Skip to Main Content
Table 2:
Question Answering Task on bAbI Data Set.
Task GORU GRU LSTM EURNN uRNN oRNN 
1 Single supporting fact 45.8 49.1 49.3 47.2 45.7 15.6 
2 Two supporting facts 39.5 38.5 32.3 24.3 18.3 18.2 
3 Three supporting facts 33.5 32.2 20.6 22.5 19.1 20.5 
4 Two argument relations 62.7 64.6 67.5 56.1 56.1 18.0 
5 Three argument relations 87.0 78.0 52.3 56.2 51.9 32.5 
6 Yes/no questions 53.6 50.5 49.3 50.5 49.2 50.1 
7 Counting 77.7 79.5 76.9 71.9 72.7 50.0 
8 Lists/sets 75.0 75.5 76.8 56.5 47.9 63.4 
9 Simple negation 62.9 63.9 63.5 60.6 61.8 63.9 
10 Indefinite knowledge 45.4 44.8 46.0 42.6 43.3 43.6 
11 Basic coreference 69.3 71.2 71.1 72.1 70.0 17.9 
12 Conjunction 69.9 71.6 71.9 72.7 71.9 16.2 
13 Compound coreference 92.7 94.2 93.8 92.4 93.2 17.2 
14 Time reasoning 37.9 39.2 34.4 20.0 23.9 20.8 
15 Basic deduction 55.2 57.4 20.9 25.0 27.1 25.4 
16 Basic induction 44.0 45.9 45.9 43.3 43.9 26.2 
17 Positional reasoning 59.6 50.5 51.6 51.2 49.5 50.6 
18 Size reasoning 90.5 89.9 91.8 89.7 86.5 51.2 
19 Path finding 8.9 9.6 8.2 9.0 7.0 10.2 
20 Agent's motivations 97.7 97.7 96.5 93.3 93.3 77.6 
Mean performance 60.4 58.2 56.0 52.9 51.6 34.5 
Task GORU GRU LSTM EURNN uRNN oRNN 
1 Single supporting fact 45.8 49.1 49.3 47.2 45.7 15.6 
2 Two supporting facts 39.5 38.5 32.3 24.3 18.3 18.2 
3 Three supporting facts 33.5 32.2 20.6 22.5 19.1 20.5 
4 Two argument relations 62.7 64.6 67.5 56.1 56.1 18.0 
5 Three argument relations 87.0 78.0 52.3 56.2 51.9 32.5 
6 Yes/no questions 53.6 50.5 49.3 50.5 49.2 50.1 
7 Counting 77.7 79.5 76.9 71.9 72.7 50.0 
8 Lists/sets 75.0 75.5 76.8 56.5 47.9 63.4 
9 Simple negation 62.9 63.9 63.5 60.6 61.8 63.9 
10 Indefinite knowledge 45.4 44.8 46.0 42.6 43.3 43.6 
11 Basic coreference 69.3 71.2 71.1 72.1 70.0 17.9 
12 Conjunction 69.9 71.6 71.9 72.7 71.9 16.2 
13 Compound coreference 92.7 94.2 93.8 92.4 93.2 17.2 
14 Time reasoning 37.9 39.2 34.4 20.0 23.9 20.8 
15 Basic deduction 55.2 57.4 20.9 25.0 27.1 25.4 
16 Basic induction 44.0 45.9 45.9 43.3 43.9 26.2 
17 Positional reasoning 59.6 50.5 51.6 51.2 49.5 50.6 
18 Size reasoning 90.5 89.9 91.8 89.7 86.5 51.2 
19 Path finding 8.9 9.6 8.2 9.0 7.0 10.2 
20 Agent's motivations 97.7 97.7 96.5 93.3 93.3 77.6 
Mean performance 60.4 58.2 56.0 52.9 51.6 34.5 

Notes: Test accuracy (%) on GORU, GRU, LSTM, EURNN, uRNN, and oRNN. All RNN models are unidirectional without extra memory or attention mechanism. GORU achieves the highest average accuracy. The bold numbers represent the highest accuracy.

Close Modal

or Create an Account

Close Modal
Close Modal