Skip to Main Content
Table 2: 
Labeled attachment scores on UD Treebanks for monolingual ([1] (Kulmizev et al., 2019) and SynTr) and multilingual (UDPipe (Straka, 2018) and UDify (Kondratyuk and Straka, 2019)) baselines, and the refined models (+RNGTr) pre-trained with BERT (Devlin et al., 2019). The relative error reduction from RNGTr refinement is shown in parentheses. Bold scores are not significantly different from the best score in that row (with α = 0.01).
LanguageTrainMonoMultiMultiMulti+MonoMonoMonoMono
Size[1]UDPipeUDifyUDify+RNGTrSynTrSynTr+RNGTrEmpty+RNGTr
Arabic 6.1K 81.8 82.94 82.88 85.93 (+17.81%) 86.23 86.31 (+0.58%) 86.05 
Basque 5.4K 79.8 82.86 80.97 87.55 (+34.57%) 87.49 88.2 (+5.68%) 87.96 
Chinese 4K 83.4 80.5 83.75 89.05 (+32.62%) 89.53 90.48 (+9.08%) 89.82 
English 12.5K 87.6 86.97 88.5 91.23 (+23.74%) 91.41 91.52 (+1.28%) 91.23 
Finnish 12.2K 83.9 87.46 82.03 91.87 (+54.76%) 91.80 91.92 (+1.46%) 91.78 
Hebrew 5.2K 85.9 86.86 88.11 90.80 (+22.62%) 91.07 91.32 (+2.79%) 90.56 
Hindi 13.3K 90.8 91.83 91.46 93.94 (+29.04%) 93.95 94.21 (+4.3%) 93.97 
Italian 13.1K 91.7 91.54 93.69 94.65 (+15.21%) 95.08 95.16 (+1.62%) 94.96 
Japanese 7.1K 92.1 93.73 92.08 95.41 (+42.06%) 95.66 95.71 (+1.16%) 95.56 
Korean 4.4K 84.2 84.24 74.26 89.12 (+57.73%) 89.29 89.45 (+1.5%) 89.1 
Russian 48.8K 91.0 92.32 93.13 94.51 (+20.09%) 94.60 94.47 (−2.4%) 94.31 
Swedish 4.3K 86.9 86.61 89.03 92.02 (+27.26%) 92.03 92.46 (+5.4%) 92.40 
Turkish 3.7K 64.9 67.56 67.44 72.07 (+14.22%) 72.52 73.08 (+2.04%) 71.99 
 
Average − 84.9 85.81 85.18 89.86 90.05 90.33 89.98 
LanguageTrainMonoMultiMultiMulti+MonoMonoMonoMono
Size[1]UDPipeUDifyUDify+RNGTrSynTrSynTr+RNGTrEmpty+RNGTr
Arabic 6.1K 81.8 82.94 82.88 85.93 (+17.81%) 86.23 86.31 (+0.58%) 86.05 
Basque 5.4K 79.8 82.86 80.97 87.55 (+34.57%) 87.49 88.2 (+5.68%) 87.96 
Chinese 4K 83.4 80.5 83.75 89.05 (+32.62%) 89.53 90.48 (+9.08%) 89.82 
English 12.5K 87.6 86.97 88.5 91.23 (+23.74%) 91.41 91.52 (+1.28%) 91.23 
Finnish 12.2K 83.9 87.46 82.03 91.87 (+54.76%) 91.80 91.92 (+1.46%) 91.78 
Hebrew 5.2K 85.9 86.86 88.11 90.80 (+22.62%) 91.07 91.32 (+2.79%) 90.56 
Hindi 13.3K 90.8 91.83 91.46 93.94 (+29.04%) 93.95 94.21 (+4.3%) 93.97 
Italian 13.1K 91.7 91.54 93.69 94.65 (+15.21%) 95.08 95.16 (+1.62%) 94.96 
Japanese 7.1K 92.1 93.73 92.08 95.41 (+42.06%) 95.66 95.71 (+1.16%) 95.56 
Korean 4.4K 84.2 84.24 74.26 89.12 (+57.73%) 89.29 89.45 (+1.5%) 89.1 
Russian 48.8K 91.0 92.32 93.13 94.51 (+20.09%) 94.60 94.47 (−2.4%) 94.31 
Swedish 4.3K 86.9 86.61 89.03 92.02 (+27.26%) 92.03 92.46 (+5.4%) 92.40 
Turkish 3.7K 64.9 67.56 67.44 72.07 (+14.22%) 72.52 73.08 (+2.04%) 71.99 
 
Average − 84.9 85.81 85.18 89.86 90.05 90.33 89.98 
Close Modal

or Create an Account

Close Modal
Close Modal