Skip to Main Content
Table 4: 

Inter-annotator agreement for our datasets calculated using Fleiss’ kappa (κ) at the token and entity level. Disagreement from type refers to the proportion of all entity-level disagreements, which are due only to type mismatch.

DatasetToken Fleiss’ κEntity Fleiss’ κDisagreement from Type
amh 0.987 0.959 0.044 
hau 0.988 0.962 0.097 
ibo 0.995 0.983 0.071 
kin 1.000 1.000 0.000 
lug 0.997 0.990 0.023 
luo 1.000 1.000 0.000 
pcm 0.989 0.966 0.048 
swa 1.000 1.000 0.000 
wol 1.000 1.000 0.000 
yor 0.990 0.964 0.079 
DatasetToken Fleiss’ κEntity Fleiss’ κDisagreement from Type
amh 0.987 0.959 0.044 
hau 0.988 0.962 0.097 
ibo 0.995 0.983 0.071 
kin 1.000 1.000 0.000 
lug 0.997 0.990 0.023 
luo 1.000 1.000 0.000 
pcm 0.989 0.966 0.048 
swa 1.000 1.000 0.000 
wol 1.000 1.000 0.000 
yor 0.990 0.964 0.079 
Close Modal

or Create an Account

Close Modal
Close Modal