Table 1

Categories of questions and their relative frequencies as identified by Clark et al. (2013). Retrieval-based questions (including is–a, dictionary definition, and property identification questions) tend to be answerable using information retrieval methods over structured knowledge bases, including taxonomies and dictionaries. More complex general inference questions make use of either simple inference rules that apply to a particular situation, a knowledge of causality, or a knowledge of simple processes (such as solids melt when heated). Difficult model-based reasoning questions require a domain-specific model of how a process works, like how gravity causes planets to orbit stars, in order to be correctly answered. Note that we do not include diagram questions, as they require specialized spatial reasoning that is beyond the scope of this work.

CategoryExample
Retrieval Q: The movement of soil by wind or water is called:
(35%) (A) condensation (B) evaporation
(C) erosion (D) friction

General Q: Which example describes an organism taking in nutrients?
Inference (A) A dog burying a bone (B) A girl eating an apple
(39%) (C) An insect crawling on a leaf (D) A boy planting tomatoes in the garden

Model-based Q: When a baby shakes a rattle, it makes a noise. Which form of energy was
Inference changed to sound energy?
(26%) (A) electrical (B) light
(C) mechanical (D) heat
CategoryExample
Retrieval Q: The movement of soil by wind or water is called:
(35%) (A) condensation (B) evaporation
(C) erosion (D) friction

General Q: Which example describes an organism taking in nutrients?
Inference (A) A dog burying a bone (B) A girl eating an apple
(39%) (C) An insect crawling on a leaf (D) A boy planting tomatoes in the garden

Model-based Q: When a baby shakes a rattle, it makes a noise. Which form of energy was
Inference changed to sound energy?
(26%) (A) electrical (B) light
(C) mechanical (D) heat
Close Modal