MIT Press

Skip to Main Content

Table 3:

Effect of NLI model choice on SummaC models performance. For each NLI model, we include the balanced accuracy scores of SummaC_ZS and SummaC_Conv. BERT X corresponds to a BERT or other pre-trained models of similar size.

Architecture	NLI Dataset	Performance
Architecture	NLI Dataset	ZS	Conv
Dec. Attn	SNLI	56.9	56.4

BERT Base	SNLI	66.6	64.0
	MNLI	69.5	69.8
	MNLI+VitaminC	67.9	71.2

BERT Large	SNLI	66.6	62.4
	SNLI+MNLI+ANLI	69.9	71.7
	VitaminC	71.1	72.8
	MNLI	70.9	73.0
	MNLI+VitaminC	72.1	74.4

Architecture	NLI Dataset	Performance
Architecture	NLI Dataset	ZS	Conv
Dec. Attn	SNLI	56.9	56.4

BERT Base	SNLI	66.6	64.0
	MNLI	69.5	69.8
	MNLI+VitaminC	67.9	71.2

BERT Large	SNLI	66.6	62.4
	SNLI+MNLI+ANLI	69.9	71.7
	VitaminC	71.1	72.8
	MNLI	70.9	73.0
	MNLI+VitaminC	72.1	74.4