Skip to Main Content
Table 6: 
Griko test set POS accuracy after each AL annotation iteration. Each iteration consists of 50 token-level annotations. The number in parentheses is the time in minutes required for annotation. The IA Agr. column reports the inter-annotator agreement against the expert linguist for the first iteration. WD is the Wasserstein distance between the selected tokens and the test distribution.
ALiteration-0iteration-1iteration-2iteration-3IA Agr.WD
Linguist-1 cral 52.93 63.42 (10) 69.07 (10) 65.16 (16) 0.58 0.281 
qbc 52.93 55.82 (15) 62.03 (17) 66.51 (15) 0.68 0.243 
uns 52.93 56.14 (15) 57.04 (15) 65.73 (11) 0.58 0.379 
 
Linguist-2 cral 52.93 61.24 (15) 67.24 (20) 67.05 (18) 0.70 0.346 
qbc 52.93 56.52 (20) 65.96 (20) 66.71 (17) 0.72 0.245 
uns 52.93 55.45 (17) 58.80 (17) 65.73 (20) 0.70 0.363 
 
Linguist-3 (Expert) cral 52.93 65.63 69.17 68.09 − 0.159 
qbc 52.93 60.50 65.69 56.20 − 0.170 
uns 52.93 58.51 64.26 65.93 − 0.125 
ALiteration-0iteration-1iteration-2iteration-3IA Agr.WD
Linguist-1 cral 52.93 63.42 (10) 69.07 (10) 65.16 (16) 0.58 0.281 
qbc 52.93 55.82 (15) 62.03 (17) 66.51 (15) 0.68 0.243 
uns 52.93 56.14 (15) 57.04 (15) 65.73 (11) 0.58 0.379 
 
Linguist-2 cral 52.93 61.24 (15) 67.24 (20) 67.05 (18) 0.70 0.346 
qbc 52.93 56.52 (20) 65.96 (20) 66.71 (17) 0.72 0.245 
uns 52.93 55.45 (17) 58.80 (17) 65.73 (20) 0.70 0.363 
 
Linguist-3 (Expert) cral 52.93 65.63 69.17 68.09 − 0.159 
qbc 52.93 60.50 65.69 56.20 − 0.170 
uns 52.93 58.51 64.26 65.93 − 0.125 
Close Modal

or Create an Account

Close Modal
Close Modal