MIT Press

Table 4:

The first category of adversarially authored questions consists of examples that require reasoning. Answer displays the correct answer (all models were incorrect). For these examples, connecting the training and adversarially authored clues is simple for humans but difficult for models.

Question	Prediction	Answer	Phenomenon
This man, who died at the Battle of the Thames, experienced a setback when his brother Tenskwatawa’s influence over their tribe began to fade.	Battle of Tippecanoe	Tecumseh	Composing Seen Clues
This number is one hundred fifty more than the number of Spartans at Thermopylae.	Battle of Thermopylae	450	Logic & Calculations
A building dedicated to this man was the site of the “I Have A Dream” speech.	Martin Luther King Jr.	Abraham Lincoln	Multi-Step Reasoning

Question	Prediction	Answer	Phenomenon
This man, who died at the Battle of the Thames, experienced a setback when his brother Tenskwatawa’s influence over their tribe began to fade.	Battle of Tippecanoe	Tecumseh	Composing Seen Clues
This number is one hundred fifty more than the number of Spartans at Thermopylae.	Battle of Thermopylae	450	Logic & Calculations
A building dedicated to this man was the site of the “I Have A Dream” speech.	Martin Luther King Jr.	Abraham Lincoln	Multi-Step Reasoning

View Large

This Feature Is Available To Subscribers Only