Table 1

Item, value, and pair weight comparisons in the case of five annotators with missing values (denoted by “.”)

item 1item 2item 3item 4
Mary noun noun noun noun
Paul noun noun noun noun
Suzan noun noun verb
Jack noun noun
Robert noun

nv = number of values
np = number of pairs 10

wu(VL) : item weight in VL 5/2 3/2
wu(PL) : item weight in PL 10

wv(IL) = 2wu(IL) / nv : value weight in IL 2/5 1/2 2/3

wv(PL) = 2wu(PL) / nv : value weight in PL

wp(IL) = wu(IL) / np : pair weight in IL 1/10 1/6 1/3
wp(VL) = wu(VL) / np : pair weight in VL 1/4 1/3 1/2

item 1item 2item 3item 4
Mary noun noun noun noun
Paul noun noun noun noun
Suzan noun noun verb
Jack noun noun
Robert noun

nv = number of values
np = number of pairs 10

wu(VL) : item weight in VL 5/2 3/2
wu(PL) : item weight in PL 10

wv(IL) = 2wu(IL) / nv : value weight in IL 2/5 1/2 2/3

wv(PL) = 2wu(PL) / nv : value weight in PL

wp(IL) = wu(IL) / np : pair weight in IL 1/10 1/6 1/3
wp(VL) = wu(VL) / np : pair weight in VL 1/4 1/3 1/2

Close Modal