Table 4:
Softmax is not a good estimate of the distribution of human labels. Exp. refers to the similarity values we expect due to random variation (i.e., what we get when we compute against a random sample drawn from the multinomial defined by the softmax). Obs. refers to the similarity values between the softmax distribution and the human distribution. Numbers in parentheses give 95% confidence intervals. Results are effectively the same for each of individual corpora, so we report only the aggregate results.
Cross Ent.Log Prob.
Exp. 0.03 (0.03, 0.03) −1.6 (−1.7, −1.5)
Obs. 0.37 (0.33, 0.42) −21.5 (−22.6, −20.1)
Cross Ent.Log Prob.
Exp. 0.03 (0.03, 0.03) −1.6 (−1.7, −1.5)
Obs. 0.37 (0.33, 0.42) −21.5 (−22.6, −20.1)
Close Modal