Many-to-Many Performance on Family Groups. We display the spBLEU on the devtest of Flores-101 for the M2M-124 615M parameter model. Each cell represents the average performance for translating from all the languages in the source group (row) into the each language of the target group (column). We highlight in gray the cells that correspond to within-group evaluation. In bold we show the best performance per target group and underline the best performance per source group.
| Num Languages: . | Afro-Asiatic . | Austronesian . | Balto-Slavic . | Bantu . | Dravidian . | Germanic . | Indo-Aryan . | Nilotic+Other AC . | Romance . | Sino-Tib+Kra-Dai . | Turkic . | Avg . |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 7 . | 6 . | 14 . | 10 . | 4 . | 9 . | 14 . | 5 . | 10 . | 4 . | 5 . | ||
| Afro-Asiatic | 4.20 | 6.82 | 10.93 | 2.31 | 1.21 | 11.95 | 3.43 | 0.93 | 11.70 | 3.66 | 2.73 | 5.44 |
| Austronesian | 6.39 | 11.50 | 13.78 | 3.48 | 2.08 | 15.53 | 4.69 | 1.45 | 14.95 | 5.78 | 4.13 | 7.61 |
| Balto-Slavic | 8.32 | 12.29 | 22.81 | 3.48 | 3.25 | 21.67 | 6.82 | 0.89 | 21.75 | 7.31 | 5.87 | 10.41 |
| Bantu | 3.28 | 5.70 | 6.29 | 2.37 | 1.16 | 7.40 | 2.16 | 1.37 | 7.16 | 2.77 | 1.95 | 3.78 |
| Dravidian | 3.04 | 4.56 | 7.21 | 1.44 | 2.34 | 7.66 | 3.62 | 0.39 | 7.31 | 2.73 | 2.04 | 3.85 |
| Germanic | 9.48 | 14.25 | 22.56 | 4.17 | 3.26 | 26.09 | 6.89 | 1.40 | 23.53 | 7.98 | 6.24 | 11.44 |
| Indo-Aryan | 3.64 | 5.27 | 8.56 | 1.60 | 2.01 | 8.81 | 3.70 | 0.42 | 8.66 | 3.26 | 2.36 | 4.39 |
| Nilotic+Other AC | 1.60 | 3.10 | 2.76 | 1.45 | 0.43 | 3.48 | 0.79 | 0.95 | 3.48 | 1.29 | 0.81 | 1.83 |
| Romance | 8.25 | 12.74 | 20.70 | 3.22 | 2.41 | 22.43 | 6.04 | 1.15 | 24.44 | 6.96 | 5.46 | 10.35 |
| Sino-Tib+Kra-Dai | 4.68 | 7.45 | 10.58 | 2.29 | 2.29 | 10.84 | 4.10 | 0.67 | 11.05 | 5.10 | 3.20 | 5.66 |
| Turkic | 3.55 | 5.24 | 9.35 | 1.61 | 1.24 | 8.81 | 2.96 | 0.58 | 9.14 | 3.13 | 2.38 | 4.36 |
| Avg | 5.13 | 8.08 | 12.32 | 2.49 | 1.97 | 13.15 | 4.11 | 0.93 | 13.01 | 4.54 | 3.38 | |
| Num Languages: . | Afro-Asiatic . | Austronesian . | Balto-Slavic . | Bantu . | Dravidian . | Germanic . | Indo-Aryan . | Nilotic+Other AC . | Romance . | Sino-Tib+Kra-Dai . | Turkic . | Avg . |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 7 . | 6 . | 14 . | 10 . | 4 . | 9 . | 14 . | 5 . | 10 . | 4 . | 5 . | ||
| Afro-Asiatic | 4.20 | 6.82 | 10.93 | 2.31 | 1.21 | 11.95 | 3.43 | 0.93 | 11.70 | 3.66 | 2.73 | 5.44 |
| Austronesian | 6.39 | 11.50 | 13.78 | 3.48 | 2.08 | 15.53 | 4.69 | 1.45 | 14.95 | 5.78 | 4.13 | 7.61 |
| Balto-Slavic | 8.32 | 12.29 | 22.81 | 3.48 | 3.25 | 21.67 | 6.82 | 0.89 | 21.75 | 7.31 | 5.87 | 10.41 |
| Bantu | 3.28 | 5.70 | 6.29 | 2.37 | 1.16 | 7.40 | 2.16 | 1.37 | 7.16 | 2.77 | 1.95 | 3.78 |
| Dravidian | 3.04 | 4.56 | 7.21 | 1.44 | 2.34 | 7.66 | 3.62 | 0.39 | 7.31 | 2.73 | 2.04 | 3.85 |
| Germanic | 9.48 | 14.25 | 22.56 | 4.17 | 3.26 | 26.09 | 6.89 | 1.40 | 23.53 | 7.98 | 6.24 | 11.44 |
| Indo-Aryan | 3.64 | 5.27 | 8.56 | 1.60 | 2.01 | 8.81 | 3.70 | 0.42 | 8.66 | 3.26 | 2.36 | 4.39 |
| Nilotic+Other AC | 1.60 | 3.10 | 2.76 | 1.45 | 0.43 | 3.48 | 0.79 | 0.95 | 3.48 | 1.29 | 0.81 | 1.83 |
| Romance | 8.25 | 12.74 | 20.70 | 3.22 | 2.41 | 22.43 | 6.04 | 1.15 | 24.44 | 6.96 | 5.46 | 10.35 |
| Sino-Tib+Kra-Dai | 4.68 | 7.45 | 10.58 | 2.29 | 2.29 | 10.84 | 4.10 | 0.67 | 11.05 | 5.10 | 3.20 | 5.66 |
| Turkic | 3.55 | 5.24 | 9.35 | 1.61 | 1.24 | 8.81 | 2.96 | 0.58 | 9.14 | 3.13 | 2.38 | 4.36 |
| Avg | 5.13 | 8.08 | 12.32 | 2.49 | 1.97 | 13.15 | 4.11 | 0.93 | 13.01 | 4.54 | 3.38 | |