Table 14
Correlation scores obtained with LessLex on different subsets of data obtained by varying standard deviation in human ratings. The reported figures show higher correlation when testing on the most reliable (with smaller standard deviation) portions of the data set. To interpret the standard deviation values, we recall that the original ratings collected in the SCWS data set were expressed in the range [0.0,10.0].
σc-rank-sim (r)rank-sim (r)nof-items
≤ 0.5 0.83 0.82 39
≤ 1.0 0.85 0.86 82
≤ 1.5 0.85 0.85 165
≤ 2.0 0.82 0.84 285
≤ 2.5 0.68 0.83 518
≤ 3.0 0.68 0.79 903
≤ 3.5 0.67 0.75 1, 429
≤ 4.0 0.64 0.71 1, 822
<5.0 0.63 0.69 2, 003
σc-rank-sim (r)rank-sim (r)nof-items
≤ 0.5 0.83 0.82 39
≤ 1.0 0.85 0.86 82
≤ 1.5 0.85 0.85 165
≤ 2.0 0.82 0.84 285
≤ 2.5 0.68 0.83 518
≤ 3.0 0.68 0.79 903
≤ 3.5 0.67 0.75 1, 429
≤ 4.0 0.64 0.71 1, 822
<5.0 0.63 0.69 2, 003
Close Modal