Skip to Main Content
Table 14 

A summary of the classes of the errors made by the system. On any given question, more than one error may have been made. The summary is broken down into three categories: incorrectly answered questions with a good justification in the top six, incorrectly answered questions without a good justification in the top six, as well as the overall proportions across these two conditions.

Error ClassGood JustificationNo Good JustificationOverall
FOCUS WORDS 
Focus Word — Question 49% 46% 48% 
Focus Word — Answer 20% 32% 27% 
Focus Word — Answer Lists 7% 5% 6% 
Focus Word — Compounds/Collocations 9% 16% 13% 
 
NOISE IN THE KNOWLEDGE BASE 
Excessively Long Sentence (Chosen Ans) 33% 9% 20% 
Excessively Long Sentence (Correct Ans) 13% 4% 8% 
 
COMPLEX INFERENCE 
More Sentences Required 4% 16% 11% 
Causal or Process Reasoning 27% 30% 29% 
Quantifiers 4% 23% 15% 
Negation 2% 9% 6% 
 
MISCELLANEOUS 
Semantic Cluster Matching 16% 5% 10% 
Coverage of Knowledge Bases 4% 36% 22% 
Other 29% 9% 18% 
Error ClassGood JustificationNo Good JustificationOverall
FOCUS WORDS 
Focus Word — Question 49% 46% 48% 
Focus Word — Answer 20% 32% 27% 
Focus Word — Answer Lists 7% 5% 6% 
Focus Word — Compounds/Collocations 9% 16% 13% 
 
NOISE IN THE KNOWLEDGE BASE 
Excessively Long Sentence (Chosen Ans) 33% 9% 20% 
Excessively Long Sentence (Correct Ans) 13% 4% 8% 
 
COMPLEX INFERENCE 
More Sentences Required 4% 16% 11% 
Causal or Process Reasoning 27% 30% 29% 
Quantifiers 4% 23% 15% 
Negation 2% 9% 6% 
 
MISCELLANEOUS 
Semantic Cluster Matching 16% 5% 10% 
Coverage of Knowledge Bases 4% 36% 22% 
Other 29% 9% 18% 
Close Modal

or Create an Account

Close Modal
Close Modal