Skip to Main Content
Table 5: 

Distribution over the implicit and multi-step properties (§2) in a sample of 100 StrategyQA questions, annotated by two experts (we average the expert decisions). Most questions are multi-step and implicit. Annotator agreement is substantial for both the implicit (κ = 0.73) and multi-step (κ = 0.65) properties.

multi-stepsingle-step
implicit 81 82 
explicit 14.5 3.5 18 
 95.5 4.5 100 
multi-stepsingle-step
implicit 81 82 
explicit 14.5 3.5 18 
 95.5 4.5 100 
Close Modal

or Create an Account

Close Modal
Close Modal