A summary of the classes of the errors made by the system. On any given question, more than one error may have been made. The summary is broken down into three categories: incorrectly answered questions with a good justification in the top six, incorrectly answered questions without a good justification in the top six, as well as the overall proportions across these two conditions.
| Error Class . | Good Justification . | No Good Justification . | Overall . |
|---|---|---|---|
| FOCUS WORDS | |||
| Focus Word — Question | 49% | 46% | 48% |
| Focus Word — Answer | 20% | 32% | 27% |
| Focus Word — Answer Lists | 7% | 5% | 6% |
| Focus Word — Compounds/Collocations | 9% | 16% | 13% |
| NOISE IN THE KNOWLEDGE BASE | |||
| Excessively Long Sentence (Chosen Ans) | 33% | 9% | 20% |
| Excessively Long Sentence (Correct Ans) | 13% | 4% | 8% |
| COMPLEX INFERENCE | |||
| More Sentences Required | 4% | 16% | 11% |
| Causal or Process Reasoning | 27% | 30% | 29% |
| Quantifiers | 4% | 23% | 15% |
| Negation | 2% | 9% | 6% |
| MISCELLANEOUS | |||
| Semantic Cluster Matching | 16% | 5% | 10% |
| Coverage of Knowledge Bases | 4% | 36% | 22% |
| Other | 29% | 9% | 18% |
| Error Class . | Good Justification . | No Good Justification . | Overall . |
|---|---|---|---|
| FOCUS WORDS | |||
| Focus Word — Question | 49% | 46% | 48% |
| Focus Word — Answer | 20% | 32% | 27% |
| Focus Word — Answer Lists | 7% | 5% | 6% |
| Focus Word — Compounds/Collocations | 9% | 16% | 13% |
| NOISE IN THE KNOWLEDGE BASE | |||
| Excessively Long Sentence (Chosen Ans) | 33% | 9% | 20% |
| Excessively Long Sentence (Correct Ans) | 13% | 4% | 8% |
| COMPLEX INFERENCE | |||
| More Sentences Required | 4% | 16% | 11% |
| Causal or Process Reasoning | 27% | 30% | 29% |
| Quantifiers | 4% | 23% | 15% |
| Negation | 2% | 9% | 6% |
| MISCELLANEOUS | |||
| Semantic Cluster Matching | 16% | 5% | 10% |
| Coverage of Knowledge Bases | 4% | 36% | 22% |
| Other | 29% | 9% | 18% |