Skip to Main Content
Table 8: 
Cluster-level accuracies (%) on the WordNetQA dev. sets for inoculated models and best Choice-only model. Δ show the absolute difference in percentage points with instance-level accuracies.
ModelDefinitionsSynonymyHypernymyHyponymy
Strict Cluster Accuracy (Δ)
Choice-Only 14.7 (−12.0) 18.5 (−22.3) 34.6 (−27.6) 4.1 (−33.7) 
 
ESIM 30.2 (−15.9) 23.3 (−26.9) 29.2 (−27.3) 15.2 (−43.8) 
BERT 68.5 (−15.5) 58.1 (−21.5) 49.0 (−24.8) 34.0 (−45.4) 
RoBERTa 75.0 (−13.9) 61.7 (−19.4) 54.0 (−23.2) 36.7 (−44.4) 
ModelDefinitionsSynonymyHypernymyHyponymy
Strict Cluster Accuracy (Δ)
Choice-Only 14.7 (−12.0) 18.5 (−22.3) 34.6 (−27.6) 4.1 (−33.7) 
 
ESIM 30.2 (−15.9) 23.3 (−26.9) 29.2 (−27.3) 15.2 (−43.8) 
BERT 68.5 (−15.5) 58.1 (−21.5) 49.0 (−24.8) 34.0 (−45.4) 
RoBERTa 75.0 (−13.9) 61.7 (−19.4) 54.0 (−23.2) 36.7 (−44.4) 
Close Modal

or Create an Account

Close Modal
Close Modal