Performance decomposition to question types of our best generative system (Gen, the best BART-based system), extractive system (Ext, the best BERT-based system, i.e., our best ranker + BERT reader), and ranker (BERT+ICT from Table 2).
| Question Type . | Ratio(%) . | QA ROUGE-L . | Ranker . | |
|---|---|---|---|---|
| Gen . | Ext . | ROUGE-L . | ||
| Relation b/w Concepts | 11.0 | 40.48 | 24.46 | 63.76 |
| Attribute of Concept | 12.0 | 34.09 | 21.69 | 56.73 |
| Event - Attribute | 3.4 | 25.88 | 10.57 | 49.23 |
| Event - Concept | 28.3 | 27.35 | 15.73 | 62.15 |
| Event - Trigger | 1.8 | 29.63 | 9.28 | 37.56 |
| Causal Relation | 12.6 | 22.86 | 10.39 | 38.47 |
| Temporal Relation | 12.6 | 28.01 | 15.57 | 49.20 |
| Nested Relation | 15.4 | 23.02 | 8.44 | 48.93 |
| Book Attribute | 2.9 | 23.11 | 25.71 | 54.60 |
| Question Type . | Ratio(%) . | QA ROUGE-L . | Ranker . | |
|---|---|---|---|---|
| Gen . | Ext . | ROUGE-L . | ||
| Relation b/w Concepts | 11.0 | 40.48 | 24.46 | 63.76 |
| Attribute of Concept | 12.0 | 34.09 | 21.69 | 56.73 |
| Event - Attribute | 3.4 | 25.88 | 10.57 | 49.23 |
| Event - Concept | 28.3 | 27.35 | 15.73 | 62.15 |
| Event - Trigger | 1.8 | 29.63 | 9.28 | 37.56 |
| Causal Relation | 12.6 | 22.86 | 10.39 | 38.47 |
| Temporal Relation | 12.6 | 28.01 | 15.57 | 49.20 |
| Nested Relation | 15.4 | 23.02 | 8.44 | 48.93 |
| Book Attribute | 2.9 | 23.11 | 25.71 | 54.60 |