Many-to-Many Performance by available Bitext through English. We show spBLEU on devtest for M2M-124 615M parameter model. spBLEU is worse for low-resource languages compared to high resource languages, and translating into low-resource languages is harder than translating out of them.
| . | Very Low . | Low . | Medium . | High . |
|---|---|---|---|---|
| < 100K . | (100K, 1M) . | (1M, 100M) . | >100M . | |
| # Langs | 15 | 40 | 38 | 6 |
| Very Low | 1.6 | 2.3 | 7.0 | 9.1 |
| Low | 2.0 | 2.74 | 8.5 | 10.3 |
| Medium | 3.8 | 5.4 | 19.1 | 23.4 |
| High | 4.3 | 5.8 | 21.7 | 27.3 |
| Avg | 2.9 | 4.1 | 14.1 | 17.6 |
| . | Very Low . | Low . | Medium . | High . |
|---|---|---|---|---|
| < 100K . | (100K, 1M) . | (1M, 100M) . | >100M . | |
| # Langs | 15 | 40 | 38 | 6 |
| Very Low | 1.6 | 2.3 | 7.0 | 9.1 |
| Low | 2.0 | 2.74 | 8.5 | 10.3 |
| Medium | 3.8 | 5.4 | 19.1 | 23.4 |
| High | 4.3 | 5.8 | 21.7 | 27.3 |
| Avg | 2.9 | 4.1 | 14.1 | 17.6 |