Skip to Main Content
Table 7: 
Models and human performance (F1 score) on the development and the test data.
In-domainOut-of-dom.In-domainOut-of-dom.
Child.Liter.Mid-High.NewsWiki.RedditScienceOverallOverallOverall
Development data 
Seq2seq 30.6 26.7 28.3 26.3 26.1 N/A N/A 27.5 N/A 27.5 
PGNet 49.7 42.4 44.8 45.5 45.0 N/A N/A 45.4 N/A 45.4 
DrQA 52.4 52.6 51.4 56.8 60.3 N/A N/A 54.7 N/A 54.7 
Augmt. DrQA 67.0 63.2 63.9 69.8 72.0 N/A N/A 67.2 N/A 67.2 
DrQA+PGNet 64.5 62.0 63.8 68.0 72.6 N/A N/A 66.2 N/A 66.2 
Human 90.7 88.3 89.1 89.9 90.9 N/A N/A 89.8 N/A 89.8 
Test data 
Seq2seq 32.8 25.6 28.0 27.0 25.3 25.6 20.1 27.7 23.0 26.3 
PGNet 49.0 43.3 47.5 47.5 45.1 38.6 38.1 46.4 38.3 44.1 
DrQA 46.7 53.9 54.1 57.8 59.4 45.0 51.0 54.5 47.9 52.6 
Augmt. DrQA 66.0 63.3 66.2 71.0 71.3 57.7 63.0 67.6 60.2 65.4 
DrQA+PGNet 64.2 63.7 67.1 68.3 71.4 57.8 63.1 67.0 60.4 65.1 
Human 90.2 88.4 89.8 88.6 89.9 86.7 88.1 89.4 87.4 88.8 
In-domainOut-of-dom.In-domainOut-of-dom.
Child.Liter.Mid-High.NewsWiki.RedditScienceOverallOverallOverall
Development data 
Seq2seq 30.6 26.7 28.3 26.3 26.1 N/A N/A 27.5 N/A 27.5 
PGNet 49.7 42.4 44.8 45.5 45.0 N/A N/A 45.4 N/A 45.4 
DrQA 52.4 52.6 51.4 56.8 60.3 N/A N/A 54.7 N/A 54.7 
Augmt. DrQA 67.0 63.2 63.9 69.8 72.0 N/A N/A 67.2 N/A 67.2 
DrQA+PGNet 64.5 62.0 63.8 68.0 72.6 N/A N/A 66.2 N/A 66.2 
Human 90.7 88.3 89.1 89.9 90.9 N/A N/A 89.8 N/A 89.8 
Test data 
Seq2seq 32.8 25.6 28.0 27.0 25.3 25.6 20.1 27.7 23.0 26.3 
PGNet 49.0 43.3 47.5 47.5 45.1 38.6 38.1 46.4 38.3 44.1 
DrQA 46.7 53.9 54.1 57.8 59.4 45.0 51.0 54.5 47.9 52.6 
Augmt. DrQA 66.0 63.3 66.2 71.0 71.3 57.7 63.0 67.6 60.2 65.4 
DrQA+PGNet 64.2 63.7 67.1 68.3 71.4 57.8 63.1 67.0 60.4 65.1 
Human 90.2 88.4 89.8 88.6 89.9 86.7 88.1 89.4 87.4 88.8 
View Large
Close Modal
This Feature Is Available To Subscribers Only

or Create an Account

Close Modal
Close Modal
This site uses cookies. By continuing to use our website, you are agreeing to our privacy policy.
Accept