Skip to Main Content
Table 5: 

mT5 vs. ByT5 on three word-level tasks. Dakshina metrics are reported on the development set to be comparable with Roark et al. (2020). Sigmorphon metrics are reported on the test sets.

ModelDakshinaSigmorphon 2020
Transliteration
CER (↓)
Grapheme-to-Phoneme
WER (↓) / PER (↓)
Inflection
Accuracy (↑)
mT5ByT5mT5ByT5mT5ByT5
Small 20.7 9.8 54.0 / 10.6 14.8 / 1.8 66.5 88.3 
Base 19.2 9.9 46.2 / 7.7 14.0 / 1.7 70.9 89.3 
Large 18.1 10.5 43.5 / 6.7 15.4 / 1.8 75.7 89.7 
XL 17.3 10.6 42.0 / 6.0 14.7 / 1.8 77.4 89.9 
XXL 16.6 9.6 40.1 / 5.4 13.8 / 1.6 78.0 90.9 
ModelDakshinaSigmorphon 2020
Transliteration
CER (↓)
Grapheme-to-Phoneme
WER (↓) / PER (↓)
Inflection
Accuracy (↑)
mT5ByT5mT5ByT5mT5ByT5
Small 20.7 9.8 54.0 / 10.6 14.8 / 1.8 66.5 88.3 
Base 19.2 9.9 46.2 / 7.7 14.0 / 1.7 70.9 89.3 
Large 18.1 10.5 43.5 / 6.7 15.4 / 1.8 75.7 89.7 
XL 17.3 10.6 42.0 / 6.0 14.7 / 1.8 77.4 89.9 
XXL 16.6 9.6 40.1 / 5.4 13.8 / 1.6 78.0 90.9 
Close Modal

or Create an Account

Close Modal
Close Modal