Skip to Main Content
Table 3: 
The QA datasets in Break. Lists the number of examples in the original dataset and in Break. Numbers of high-level QDMRs are denoted by high.
DatasetExampleOriginalBreak
Academic (DB) Return me the total citations of all the papers in the VLDB conference. 195 195 
ATIS (DB) What is the first flight from Atlanta to Baltimore that serves lunch? 5,283 4,906 
GeoQuery (DB) How high is the highest point in the largest state? 880 877 
Spider (DB) How many transactions correspond to each invoice number? 10,181 7,982 
CLEVR-humans (Images) What is the number of cylinders divided by the number of cubes? 32,164 13,935 
NLVR2 (ImagesIf there are only two dogs pulling one of the sleds? 29,680 13,517 
ComQA (TextWhat was Gandhi’s occupation before becoming a freedom fighter? 11,214 5,520 
CWQ (TextRobert E Jordan is part of the organization started by whom? 34,689 2,988, 2,991high 
DROP (TextApproximately how many years did the churches built in 1909 survive? 96,567 10,230, 10,262high 
HotpotQA-hard (TextBenjamin Halfpenny was a footballer for a club that plays its home matches where? 23,066 10,575high 
 
Total:   83,978 
DatasetExampleOriginalBreak
Academic (DB) Return me the total citations of all the papers in the VLDB conference. 195 195 
ATIS (DB) What is the first flight from Atlanta to Baltimore that serves lunch? 5,283 4,906 
GeoQuery (DB) How high is the highest point in the largest state? 880 877 
Spider (DB) How many transactions correspond to each invoice number? 10,181 7,982 
CLEVR-humans (Images) What is the number of cylinders divided by the number of cubes? 32,164 13,935 
NLVR2 (ImagesIf there are only two dogs pulling one of the sleds? 29,680 13,517 
ComQA (TextWhat was Gandhi’s occupation before becoming a freedom fighter? 11,214 5,520 
CWQ (TextRobert E Jordan is part of the organization started by whom? 34,689 2,988, 2,991high 
DROP (TextApproximately how many years did the churches built in 1909 survive? 96,567 10,230, 10,262high 
HotpotQA-hard (TextBenjamin Halfpenny was a footballer for a club that plays its home matches where? 23,066 10,575high 
 
Total:   83,978 
Close Modal

or Create an Account

Close Modal
Close Modal