Skip to Main Content
Table 8

Evaluation on the top quartile of most associated pairs in SpA-Verb (Top Q), compared against Spearman’s correlation scores on the whole data set (SpA-Verb), and on the top 3 classes with the highest IAA (ρ > 0.50) (#3, #1, #13).

ModelSpA-VerbTop Q#3#1#13
SGNS-BOW2 0.179 0.069 0.082 0.045 0.108 
  
CBOW-CC 0.271 0.216 0.084 0.322 0.378 
ELMo-Static 0.230 0.237 0.098 0.114 0.238 
  
Non-distributional 0.295 0.450 0.386 0.359 0.480 
SGNS + Attract-Repel 0.385 0.526 0.718 0.338 0.534 
BOW2-VN 0.176 0.205 −0.037 0.198 0.212 
BOW2-FN 0.210 0.287 0.095 0.242 0.156 
  
BERT-BASE (iso0.235 0.181 0.123 0.201 0.330 
context-10 0.262 0.181 0.154 0.177 0.366 
context-100 0.265 0.183 0.149 0.193 0.380 
context-500 0.265 0.185 0.148 0.190 0.379 
  
BERT-LARGE (iso0.224 0.152 0.164 0.245 0.246 
context-10 0.255 0.163 0.180 0.242 0.318 
context-100 0.256 0.167 0.172 0.245 0.326 
context-500 0.256 0.167 0.171 0.245 0.326 
  
BERT-LARGE-WWM (iso0.237 0.195 0.170 0.215 0.276 
ModelSpA-VerbTop Q#3#1#13
SGNS-BOW2 0.179 0.069 0.082 0.045 0.108 
  
CBOW-CC 0.271 0.216 0.084 0.322 0.378 
ELMo-Static 0.230 0.237 0.098 0.114 0.238 
  
Non-distributional 0.295 0.450 0.386 0.359 0.480 
SGNS + Attract-Repel 0.385 0.526 0.718 0.338 0.534 
BOW2-VN 0.176 0.205 −0.037 0.198 0.212 
BOW2-FN 0.210 0.287 0.095 0.242 0.156 
  
BERT-BASE (iso0.235 0.181 0.123 0.201 0.330 
context-10 0.262 0.181 0.154 0.177 0.366 
context-100 0.265 0.183 0.149 0.193 0.380 
context-500 0.265 0.185 0.148 0.190 0.379 
  
BERT-LARGE (iso0.224 0.152 0.164 0.245 0.246 
context-10 0.255 0.163 0.180 0.242 0.318 
context-100 0.256 0.167 0.172 0.245 0.326 
context-500 0.256 0.167 0.171 0.245 0.326 
  
BERT-LARGE-WWM (iso0.237 0.195 0.170 0.215 0.276 
Close Modal

or Create an Account

Close Modal
Close Modal