Skip to Main Content
Table 3: 

Effect of NLI model choice on SummaC models performance. For each NLI model, we include the balanced accuracy scores of SummaCZS and SummaCConv. BERT X corresponds to a BERT or other pre-trained models of similar size.

ArchitectureNLI DatasetPerformance
ZSConv
Dec. Attn SNLI 56.9 56.4 
 
BERT Base SNLI 66.6 64.0 
MNLI 69.5 69.8 
MNLI+VitaminC 67.9 71.2 
 
BERT Large SNLI 66.6 62.4 
SNLI+MNLI+ANLI 69.9 71.7 
VitaminC 71.1 72.8 
MNLI 70.9 73.0 
MNLI+VitaminC 72.1 74.4 
ArchitectureNLI DatasetPerformance
ZSConv
Dec. Attn SNLI 56.9 56.4 
 
BERT Base SNLI 66.6 64.0 
MNLI 69.5 69.8 
MNLI+VitaminC 67.9 71.2 
 
BERT Large SNLI 66.6 62.4 
SNLI+MNLI+ANLI 69.9 71.7 
VitaminC 71.1 72.8 
MNLI 70.9 73.0 
MNLI+VitaminC 72.1 74.4 
Close Modal

or Create an Account

Close Modal
Close Modal