mGENRE on the Mewsli-9. Models are trained only on the Mewsli-9 languages (1M datapoints per language). ‘Can.’ is canonical, ‘N+L’ is ‘name+language‘ and ‘L+N’ is the opposite. M indicates marginalization.
| Lang. . | Can. . | N+L . | L+N . | L+NM . |
|---|---|---|---|---|
| ar | 90.5 | 92.8 | 92.9 | 89.2 |
| de | 84.6 | 86.4 | 86.4 | 85.3 |
| en | 77.6 | 79.3 | 79.2 | 76.5 |
| es | 83.4 | 85.5 | 85.2 | 83.4 |
| fa | 91.6 | 90.7 | 91.8 | 88.2 |
| ja | 81.3 | 82.3 | 82.8 | 81.3 |
| sr | 91.5 | 92.7 | 92.9 | 92.5 |
| ta | 92.8 | 91.8 | 91.9 | 91.3 |
| tr | 88.0 | 87.7 | 87.3 | 86.0 |
| micro-avg | 83.20 | 84.77 | 84.80 | 83.05 |
| macro-avg | 86.82 | 87.68 | 87.82 | 85.97 |
| + candidates | ||||
| ar | 94.4 | 94.5 | 94.7 | 93.0 |
| de | 89.4 | 89.8 | 89.8 | 89.3 |
| en | 83.6 | 83.8 | 83.9 | 82.4 |
| es | 87.7 | 88.2 | 88.3 | 87.3 |
| fa | 93.6 | 93.3 | 93.6 | 93.3 |
| ja | 87.9 | 88.0 | 88.4 | 87.9 |
| sr | 93.1 | 93.4 | 93.5 | 93.2 |
| ta | 93.0 | 92.2 | 92.5 | 92.5 |
| tr | 91.1 | 90.4 | 89.9 | 89.1 |
| micro-avg | 87.95 | 88.22 | 88.32 | 87.43 |
| macro-avg | 90.42 | 90.41 | 90.51 | 89.78 |
| Lang. . | Can. . | N+L . | L+N . | L+NM . |
|---|---|---|---|---|
| ar | 90.5 | 92.8 | 92.9 | 89.2 |
| de | 84.6 | 86.4 | 86.4 | 85.3 |
| en | 77.6 | 79.3 | 79.2 | 76.5 |
| es | 83.4 | 85.5 | 85.2 | 83.4 |
| fa | 91.6 | 90.7 | 91.8 | 88.2 |
| ja | 81.3 | 82.3 | 82.8 | 81.3 |
| sr | 91.5 | 92.7 | 92.9 | 92.5 |
| ta | 92.8 | 91.8 | 91.9 | 91.3 |
| tr | 88.0 | 87.7 | 87.3 | 86.0 |
| micro-avg | 83.20 | 84.77 | 84.80 | 83.05 |
| macro-avg | 86.82 | 87.68 | 87.82 | 85.97 |
| + candidates | ||||
| ar | 94.4 | 94.5 | 94.7 | 93.0 |
| de | 89.4 | 89.8 | 89.8 | 89.3 |
| en | 83.6 | 83.8 | 83.9 | 82.4 |
| es | 87.7 | 88.2 | 88.3 | 87.3 |
| fa | 93.6 | 93.3 | 93.6 | 93.3 |
| ja | 87.9 | 88.0 | 88.4 | 87.9 |
| sr | 93.1 | 93.4 | 93.5 | 93.2 |
| ta | 93.0 | 92.2 | 92.5 | 92.5 |
| tr | 91.1 | 90.4 | 89.9 | 89.1 |
| micro-avg | 87.95 | 88.22 | 88.32 | 87.43 |
| macro-avg | 90.42 | 90.41 | 90.51 | 89.78 |