Language-wise breakdown for TyDi QA primary tasks. English is parenthesized because it is not included in the overall score calculation for TyDi QA.
| Language . | mBert . | Canine-S . | Canine-C . | Canine-C + n-grams . |
|---|---|---|---|---|
| SelectP | ||||
| (English) | 62.2 | 58.6 (–3.6) | 61.6 (–0.6) | 64.6 (+2.4) |
| Arabic | 82.3 | 82.8 (+0.5) | 82.5 (+0.2) | 84.3 (+2.0) |
| Bengali | 58.5 | 61.8 (+3.3) | 62.5 (+4.0) | 66.0 (+7.5) |
| Finnish | 60.4 | 62.2 (+1.8) | 63.6 (+3.2) | 66.7 (+6.3) |
| Indonesian | 61.3 | 63.5 (+2.2) | 64.2 (+2.9) | 65.9 (+4.6) |
| Japanese | 46.2 | 51.7 (+5.5) | 49.7 (+3.5) | 51.2 (+5.0) |
| Korean | 60.2 | 60.3 (+0.1) | 59.7 (–0.5) | 60.6 (+0.4) |
| Russian | 62.2 | 64.6 (+2.4) | 65.6 (+3.4) | 68.5 (+6.3) |
| Swahili | 58.8 | 67.8 (+9.0) | 67.0 (+8.2) | 67.2 (+8.4) |
| Telugu | 81.0 | 82.5 (+1.5) | 81.1 (+0.1) | 84.6 (+3.6) |
| Thai | 61.1 | 62.8 (+1.7) | 61.2 (+0.1) | 65.8 (+4.7) |
| Macro Avg | 63.2 | 66.0 (+2.8) | 65.7 (+2.5) | 68.1 (+4.9) |
| MinSpan | ||||
| (English) | 46.0 | 46.3 (+0.3) | 49.0 (+3.0) | 51.8 (+5.8) |
| Arabic | 70.7 | 66.9 (–3.8) | 65.6 (–5.1) | 73.0 (+2.3) |
| Bengali | 47.3 | 46.7 (–0.6) | 52.5 (+5.2) | 57.1 (+9.8) |
| Finnish | 51.1 | 53.0 (+1.9) | 53.8 (+2.7) | 57.1 (+6.0) |
| Indonesian | 52.2 | 53.6 (+1.4) | 54.4 (+2.2) | 56.8 (+4.6) |
| Japanese | 36.1 | 40.3 (+4.2) | 40.7 (+4.6) | 42.0 (+5.9) |
| Korean | 36.8 | 35.7 (–1.1) | 36.5 (–0.3) | 39.9 (+3.1) |
| Russian | 45.6 | 46.7 (+1.1) | 47.2 (+1.6) | 51.5 (+5.9) |
| Swahili | 49.4 | 59.0 (+9.6) | 57.6 (+8.2) | 59.2 (+9.8) |
| Telugu | 75.6 | 75.2 (–0.4) | 74.2 (–1.4) | 79.7 (+4.1) |
| Thai | 48.4 | 47.9 (–0.5) | 47.1 (–1.3) | 54.2 (+5.8) |
| Macro Avg | 51.3 | 52.5 (+1.2) | 53.0 (+1.7) | 57.0 (+5.7) |
| Language . | mBert . | Canine-S . | Canine-C . | Canine-C + n-grams . |
|---|---|---|---|---|
| SelectP | ||||
| (English) | 62.2 | 58.6 (–3.6) | 61.6 (–0.6) | 64.6 (+2.4) |
| Arabic | 82.3 | 82.8 (+0.5) | 82.5 (+0.2) | 84.3 (+2.0) |
| Bengali | 58.5 | 61.8 (+3.3) | 62.5 (+4.0) | 66.0 (+7.5) |
| Finnish | 60.4 | 62.2 (+1.8) | 63.6 (+3.2) | 66.7 (+6.3) |
| Indonesian | 61.3 | 63.5 (+2.2) | 64.2 (+2.9) | 65.9 (+4.6) |
| Japanese | 46.2 | 51.7 (+5.5) | 49.7 (+3.5) | 51.2 (+5.0) |
| Korean | 60.2 | 60.3 (+0.1) | 59.7 (–0.5) | 60.6 (+0.4) |
| Russian | 62.2 | 64.6 (+2.4) | 65.6 (+3.4) | 68.5 (+6.3) |
| Swahili | 58.8 | 67.8 (+9.0) | 67.0 (+8.2) | 67.2 (+8.4) |
| Telugu | 81.0 | 82.5 (+1.5) | 81.1 (+0.1) | 84.6 (+3.6) |
| Thai | 61.1 | 62.8 (+1.7) | 61.2 (+0.1) | 65.8 (+4.7) |
| Macro Avg | 63.2 | 66.0 (+2.8) | 65.7 (+2.5) | 68.1 (+4.9) |
| MinSpan | ||||
| (English) | 46.0 | 46.3 (+0.3) | 49.0 (+3.0) | 51.8 (+5.8) |
| Arabic | 70.7 | 66.9 (–3.8) | 65.6 (–5.1) | 73.0 (+2.3) |
| Bengali | 47.3 | 46.7 (–0.6) | 52.5 (+5.2) | 57.1 (+9.8) |
| Finnish | 51.1 | 53.0 (+1.9) | 53.8 (+2.7) | 57.1 (+6.0) |
| Indonesian | 52.2 | 53.6 (+1.4) | 54.4 (+2.2) | 56.8 (+4.6) |
| Japanese | 36.1 | 40.3 (+4.2) | 40.7 (+4.6) | 42.0 (+5.9) |
| Korean | 36.8 | 35.7 (–1.1) | 36.5 (–0.3) | 39.9 (+3.1) |
| Russian | 45.6 | 46.7 (+1.1) | 47.2 (+1.6) | 51.5 (+5.9) |
| Swahili | 49.4 | 59.0 (+9.6) | 57.6 (+8.2) | 59.2 (+9.8) |
| Telugu | 75.6 | 75.2 (–0.4) | 74.2 (–1.4) | 79.7 (+4.1) |
| Thai | 48.4 | 47.9 (–0.5) | 47.1 (–1.3) | 54.2 (+5.8) |
| Macro Avg | 51.3 | 52.5 (+1.2) | 53.0 (+1.7) | 57.0 (+5.7) |