Skip to Main Content
Table 11 
Token-Level probing task results for all languages. Bold represents the best score, and italics represents the second best.
Finnish
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Case 35.9 79.7 69.1 85.0 97.3 97.9 98 
Mood 89.7 94.9 94.3 96.7 97.4 97.3 98.3 
Number 82.2 92.8 90.4 94.4 97.8 98.3 98.7 
POS 29.5 69.0 72.0 68.9 71.3 74.8 87.5 
Person 64.5 92.8 89.7 95.1 97.3 96.8 96.4 
Tense 62.9 95.2 92.8 97.5 98.4 98.4 98.1 
Voice 86.2 96.2 92.7 96.5 97.7 98.0 97.5 
German
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Case 32.7 56.9 52.9 53.0 56.8 53.4 80.8 
Gender 38.6 72.6 69.3 66.5 73.5 72.8 78.7 
Mood 96.4 98.3 98.4 98.4 98.9 98.7 98.9 
Number 79.5 88.5 88.1 86.1 89.5 89.2 94.7 
POS 20.6 76.5 82.6 74.5 76.4 77.1 91.2 
Person 72.8 94.6 94.1 93.8 95.1 95.1 97.9 
Tense 51.6 98.7 97.8 98.7 98.6 99.2 98.8 
Spanish
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Gender 58.0 98.6 95.3 97.9 99.2 99.3 99.4 
Mood 92.3 97.7 97.2 95.9 99.1 98.2 97.8 
Number 75.2 98.7 96.4 98.6 99.6 99.6 99.6 
POS 19.3 78.1 83.2 78.0 79.6 78.9 92 
Person 68.4 97.4 95.3 98.1 99.2 99.0 99.0 
Tense 55.4 98.2 97.1 98.7 99.3 99.2 99.1 
Russian
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Case 31.7 75.3 47.6 68.2 78.6 78.7 89.7 
Gender 45.4 86.0 64.7 87.0 89.9 89.8 89.7 
Number 75.3 93.9 78.2 92.8 96.8 97.0 97.4 
POS 30.3 76.4 64.5 73.3 76.5 76.9 95.4 
Person 51.1 91.8 84.3 98.1 98.6 98.6 98.8 
Tense 53.0 90.6 78.4 97.6 98.7 97.2 96.9 
Voice 69.5 88.9 79.0 95.6 93.7 95.4 94.5 
Turkish
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Case 47.7 84.3 67.7 89.0 91.4 95.4 96.6 
POS 35.4 71.7 68.0 75.4 78.3 77.4 83.2 
Person 78.3 91.0 84.3 95.0 97.2 96.8 96.5 
Polarity 89.0 95.4 92.3 98.3 99.2 99.0 98.8 
Tense 55.2 87.7 76.3 94.6 96.6 95.9 96.3 
Finnish
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Case 35.9 79.7 69.1 85.0 97.3 97.9 98 
Mood 89.7 94.9 94.3 96.7 97.4 97.3 98.3 
Number 82.2 92.8 90.4 94.4 97.8 98.3 98.7 
POS 29.5 69.0 72.0 68.9 71.3 74.8 87.5 
Person 64.5 92.8 89.7 95.1 97.3 96.8 96.4 
Tense 62.9 95.2 92.8 97.5 98.4 98.4 98.1 
Voice 86.2 96.2 92.7 96.5 97.7 98.0 97.5 
German
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Case 32.7 56.9 52.9 53.0 56.8 53.4 80.8 
Gender 38.6 72.6 69.3 66.5 73.5 72.8 78.7 
Mood 96.4 98.3 98.4 98.4 98.9 98.7 98.9 
Number 79.5 88.5 88.1 86.1 89.5 89.2 94.7 
POS 20.6 76.5 82.6 74.5 76.4 77.1 91.2 
Person 72.8 94.6 94.1 93.8 95.1 95.1 97.9 
Tense 51.6 98.7 97.8 98.7 98.6 99.2 98.8 
Spanish
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Gender 58.0 98.6 95.3 97.9 99.2 99.3 99.4 
Mood 92.3 97.7 97.2 95.9 99.1 98.2 97.8 
Number 75.2 98.7 96.4 98.6 99.6 99.6 99.6 
POS 19.3 78.1 83.2 78.0 79.6 78.9 92 
Person 68.4 97.4 95.3 98.1 99.2 99.0 99.0 
Tense 55.4 98.2 97.1 98.7 99.3 99.2 99.1 
Russian
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Case 31.7 75.3 47.6 68.2 78.6 78.7 89.7 
Gender 45.4 86.0 64.7 87.0 89.9 89.8 89.7 
Number 75.3 93.9 78.2 92.8 96.8 97.0 97.4 
POS 30.3 76.4 64.5 73.3 76.5 76.9 95.4 
Person 51.1 91.8 84.3 98.1 98.6 98.6 98.8 
Tense 53.0 90.6 78.4 97.6 98.7 97.2 96.9 
Voice 69.5 88.9 79.0 95.6 93.7 95.4 94.5 
Turkish
TaskbaselineMUSEword2vecGloVe-BPEfastTextD-ELMoC-ELMo
Case 47.7 84.3 67.7 89.0 91.4 95.4 96.6 
POS 35.4 71.7 68.0 75.4 78.3 77.4 83.2 
Person 78.3 91.0 84.3 95.0 97.2 96.8 96.5 
Polarity 89.0 95.4 92.3 98.3 99.2 99.0 98.8 
Tense 55.2 87.7 76.3 94.6 96.6 95.9 96.3 
Close Modal

or Create an Account

Close Modal
Close Modal