Skip to Main Content
Table 7: 
Training models on SQuAD, as well as SQuAD combined with different adversarially created datasets. Results underlined indicate the best result per model. We report the mean and standard deviation (subscript) over 10 runs with different random seeds.
Evaluation (Test) Dataset
ModelTraining DatasetDSQuADDBiDAFDBERTDRoBERTa
EMF1EMF1EMF1EMF1
BiDAF DSQuAD 56.70.5 70.10.3 11.61.0 21.31.1 8.60.6 17.30.8 8.30.7 16.80.5 
DSQuAD + DBiDAF 56.30.6 69.70.4 14.40.9 24.40.9 15.61.1 24.71.1 14.30.5 23.30.7 
DSQuAD + DBERT 56.20.6 69.40.6 14.40.7 24.20.8 15.70.6 25.10.6 13.90.8 22.70.8 
DSQuAD + DRoBERTa 56.20.7 69.60.6 14.70.9 24.80.8 17.90.5 26.70.6 16.71.1 25.00.8 
 
BERT DSQuAD 74.80.3 86.90.2 46.40.7 60.50.8 24.41.2 35.91.1 17.30.7 28.90.9 
DSQuAD + DBiDAF 75.20.4 87.20.2 52.40.9 66.50.9 40.91.3 51.21.5 32.90.9 44.10.8 
DSQuAD + DBERT 75.10.3 87.10.3 54.11.0 68.00.8 43.71.1 54.11.3 34.70.7 45.70.8 
DSQuAD + DRoBERTa 75.30.4 87.10.3 53.01.1 67.10.8 44.11.1 54.40.9 36.60.8 47.80.5 
 
RoBERTa DSQuAD 73.20.4 86.30.2 48.91.1 64.31.1 31.31.1 43.51.2 16.10.8 26.70.9 
DSQuAD + DBiDAF 73.90.4 86.70.2 55.01.4 69.70.9 46.51.1 57.31.1 31.90.8 42.41.0 
DSQuAD + DBERT 73.80.2 86.70.2 55.41.0 70.10.9 48.91.0 59.01.2 32.91.3 43.71.4 
DSQuAD + DRoBERTa 73.50.3 86.50.2 55.90.7 70.60.7 49.11.2 59.51.2 34.71.0 45.91.2 
Evaluation (Test) Dataset
ModelTraining DatasetDSQuADDBiDAFDBERTDRoBERTa
EMF1EMF1EMF1EMF1
BiDAF DSQuAD 56.70.5 70.10.3 11.61.0 21.31.1 8.60.6 17.30.8 8.30.7 16.80.5 
DSQuAD + DBiDAF 56.30.6 69.70.4 14.40.9 24.40.9 15.61.1 24.71.1 14.30.5 23.30.7 
DSQuAD + DBERT 56.20.6 69.40.6 14.40.7 24.20.8 15.70.6 25.10.6 13.90.8 22.70.8 
DSQuAD + DRoBERTa 56.20.7 69.60.6 14.70.9 24.80.8 17.90.5 26.70.6 16.71.1 25.00.8 
 
BERT DSQuAD 74.80.3 86.90.2 46.40.7 60.50.8 24.41.2 35.91.1 17.30.7 28.90.9 
DSQuAD + DBiDAF 75.20.4 87.20.2 52.40.9 66.50.9 40.91.3 51.21.5 32.90.9 44.10.8 
DSQuAD + DBERT 75.10.3 87.10.3 54.11.0 68.00.8 43.71.1 54.11.3 34.70.7 45.70.8 
DSQuAD + DRoBERTa 75.30.4 87.10.3 53.01.1 67.10.8 44.11.1 54.40.9 36.60.8 47.80.5 
 
RoBERTa DSQuAD 73.20.4 86.30.2 48.91.1 64.31.1 31.31.1 43.51.2 16.10.8 26.70.9 
DSQuAD + DBiDAF 73.90.4 86.70.2 55.01.4 69.70.9 46.51.1 57.31.1 31.90.8 42.41.0 
DSQuAD + DBERT 73.80.2 86.70.2 55.41.0 70.10.9 48.91.0 59.01.2 32.91.3 43.71.4 
DSQuAD + DRoBERTa 73.50.3 86.50.2 55.90.7 70.60.7 49.11.2 59.51.2 34.71.0 45.91.2 
Close Modal

or Create an Account

Close Modal
Close Modal