Skip to Main Content
Table A2 
Conditional Spearman correlation with direct assessment scores for popular and top scoring metrics from the WMT16 Metrics Task for a four-way split of the data set resulting in data samples corresponding to four quality levels (Q1Q4). This table corresponds to Table 4 in the main body of the article.
Q1Q2Q3Q4Q4*All
Meteor 0.176 0.154 0.142 0.399 0.303 0.565 
-TERp-A 0.157 0.107 0.180 0.384 0.286 0.554 
MPEDA 0.177 0.153 0.146 0.393 0.297 0.563 
ROUGE-SU* 0.178 0.109 0.186 0.354 0.258 0.521 
ChrF3 0.185 0.128 0.139 0.345 0.243 0.515 
NIST-4 0.158 0.105 0.141 0.361 0.263 0.478 
BLEU-4 0.051 0.065 0.133 0.364 0.264 0.456 
-TER 0.044 0.062 0.183 0.371 0.273 0.455 
-WER 0.016 0.056 0.194 0.389 0.293 0.443 
-PER 0.110 0.091 0.140 0.350 0.248 0.429 
 
UPF-Cobalt 0.139 0.118 0.161 0.388 0.293 0.544 
CP-Oc(*) 0.147 0.071 0.171 0.358 0.258 0.491 
SP-lNIST 0.166 0.105 0.142 0.359 0.260 0.483 
DP-Oc(*) 0.036 0.044 0.144 0.302 0.213 0.385 
SR-Or(*) 0.098 0.081 0.042 0.171 0.173 0.307 
 
DPMFcomb 0.178 0.139 0.186 0.405 0.312 0.589 
Metrics-F 0.124 0.164 0.192 0.413 0.321 0.590 
Cobalt-F-comp 0.098 0.150 0.203 0.423 0.332 0.586 
BEER 0.194 0.123 0.147 0.340 0.238 0.505 
UoW-ReVal 0.111 0.097 0.165 0.380 0.282 0.507 
Q1Q2Q3Q4Q4*All
Meteor 0.176 0.154 0.142 0.399 0.303 0.565 
-TERp-A 0.157 0.107 0.180 0.384 0.286 0.554 
MPEDA 0.177 0.153 0.146 0.393 0.297 0.563 
ROUGE-SU* 0.178 0.109 0.186 0.354 0.258 0.521 
ChrF3 0.185 0.128 0.139 0.345 0.243 0.515 
NIST-4 0.158 0.105 0.141 0.361 0.263 0.478 
BLEU-4 0.051 0.065 0.133 0.364 0.264 0.456 
-TER 0.044 0.062 0.183 0.371 0.273 0.455 
-WER 0.016 0.056 0.194 0.389 0.293 0.443 
-PER 0.110 0.091 0.140 0.350 0.248 0.429 
 
UPF-Cobalt 0.139 0.118 0.161 0.388 0.293 0.544 
CP-Oc(*) 0.147 0.071 0.171 0.358 0.258 0.491 
SP-lNIST 0.166 0.105 0.142 0.359 0.260 0.483 
DP-Oc(*) 0.036 0.044 0.144 0.302 0.213 0.385 
SR-Or(*) 0.098 0.081 0.042 0.171 0.173 0.307 
 
DPMFcomb 0.178 0.139 0.186 0.405 0.312 0.589 
Metrics-F 0.124 0.164 0.192 0.413 0.321 0.590 
Cobalt-F-comp 0.098 0.150 0.203 0.423 0.332 0.586 
BEER 0.194 0.123 0.147 0.340 0.238 0.505 
UoW-ReVal 0.111 0.097 0.165 0.380 0.282 0.507 
Close Modal

or Create an Account

Close Modal
Close Modal