mT5 vs. ByT5 on three word-level tasks. Dakshina metrics are reported on the development set to be comparable with Roark et al. (2020). Sigmorphon metrics are reported on the test sets.
| Model . | Dakshina . | Sigmorphon 2020 . | ||||
|---|---|---|---|---|---|---|
| Transliteration CER (↓) . | Grapheme-to-Phoneme WER (↓) / PER (↓) . | Inflection Accuracy (↑) . | ||||
| mT5 . | ByT5 . | mT5 . | ByT5 . | mT5 . | ByT5 . | |
| Small | 20.7 | 9.8 | 54.0 / 10.6 | 14.8 / 1.8 | 66.5 | 88.3 |
| Base | 19.2 | 9.9 | 46.2 / 7.7 | 14.0 / 1.7 | 70.9 | 89.3 |
| Large | 18.1 | 10.5 | 43.5 / 6.7 | 15.4 / 1.8 | 75.7 | 89.7 |
| XL | 17.3 | 10.6 | 42.0 / 6.0 | 14.7 / 1.8 | 77.4 | 89.9 |
| XXL | 16.6 | 9.6 | 40.1 / 5.4 | 13.8 / 1.6 | 78.0 | 90.9 |
| Model . | Dakshina . | Sigmorphon 2020 . | ||||
|---|---|---|---|---|---|---|
| Transliteration CER (↓) . | Grapheme-to-Phoneme WER (↓) / PER (↓) . | Inflection Accuracy (↑) . | ||||
| mT5 . | ByT5 . | mT5 . | ByT5 . | mT5 . | ByT5 . | |
| Small | 20.7 | 9.8 | 54.0 / 10.6 | 14.8 / 1.8 | 66.5 | 88.3 |
| Base | 19.2 | 9.9 | 46.2 / 7.7 | 14.0 / 1.7 | 70.9 | 89.3 |
| Large | 18.1 | 10.5 | 43.5 / 6.7 | 15.4 / 1.8 | 75.7 | 89.7 |
| XL | 17.3 | 10.6 | 42.0 / 6.0 | 14.7 / 1.8 | 77.4 | 89.9 |
| XXL | 16.6 | 9.6 | 40.1 / 5.4 | 13.8 / 1.6 | 78.0 | 90.9 |