Skip to Main Content
Table 4: 

Effect of NLI category inclusion on SummaCConv performance. Models had access to different subsets of the three category predictions (Entailment, Neutral, Contradiction), with performance measured in terms of balanced accuracy. Experiments were performed with 3 NLI models: Vitamic C+MNLI, ANLI, and MNLI.

CategorySummaCConv Performance
ENCVITC+MNLIANLIMNLI
✓   74.4 69.2 72.6 
 ✓  71.2 55.8 66.4 
  ✓ 72.5 69.2 72.6 
✓ ✓  73.1 69.6 72.6 
✓  ✓ 74.0 70.2 73.0 
 ✓ ✓ 72.5 69.2 72.6 
✓ ✓ ✓ 74.0 69.7 73.0 
CategorySummaCConv Performance
ENCVITC+MNLIANLIMNLI
✓   74.4 69.2 72.6 
 ✓  71.2 55.8 66.4 
  ✓ 72.5 69.2 72.6 
✓ ✓  73.1 69.6 72.6 
✓  ✓ 74.0 70.2 73.0 
 ✓ ✓ 72.5 69.2 72.6 
✓ ✓ ✓ 74.0 69.7 73.0 
Close Modal

or Create an Account

Close Modal
Close Modal