MIT Press

Table 3:

F1 score (%) of KVMnet and NSM, and CIPITR. Bold numbers indicate the best among KVMnet and top beam score of NSM and CIPITR.

↓ Run name ∖ Question type →	Simple	Logical	Verify	Quanti.	Quant Count	Compar.	Comp Count	All
Training Size Stats.	462K	93K	43K	99K	122K	41K	42K	904K
Test Size Stats.	81K	18K	9K	9K	18K	7K	7K	150K
KVMnet	41.40	37.56	27.28	0.89	17.80	1.63	9.60	26.67
NSM, best at top beam	78.38	35.40	28.70	4.31	12.38	0.17	0.00	10.63
NSM best over top 2 beams	80.12	41.23	35.67	4.65	15.34	0.21	0.00	11.02
NSM, best over top 5 beams	86.46	64.70	50.80	6.98	29.18	0.48	0.00	12.07
NSM, best over top 10 beams	96.78	69.86	60.18	10.69	30.71	2.09	0.00	14.36
CIPITR, best at top beam	96.52	87.72	89.43	23.91	51.33	15.12	0.33	58.92
CIPITR, best over top 2 beams	96.55	87.78	90.48	25.85	51.72	19.85	0.41	62.52
CIPITR, best over top 5 beams	97.18	87.96	90.97	27.19	52.01	29.45	1.01	69.25
CIPITR, best over top 10 beams	97.18	88.92	90.98	28.92	52.71	32.98	1.54	73.71

↓ Run name ∖ Question type →	Simple	Logical	Verify	Quanti.	Quant Count	Compar.	Comp Count	All
Training Size Stats.	462K	93K	43K	99K	122K	41K	42K	904K
Test Size Stats.	81K	18K	9K	9K	18K	7K	7K	150K
KVMnet	41.40	37.56	27.28	0.89	17.80	1.63	9.60	26.67
NSM, best at top beam	78.38	35.40	28.70	4.31	12.38	0.17	0.00	10.63
NSM best over top 2 beams	80.12	41.23	35.67	4.65	15.34	0.21	0.00	11.02
NSM, best over top 5 beams	86.46	64.70	50.80	6.98	29.18	0.48	0.00	12.07
NSM, best over top 10 beams	96.78	69.86	60.18	10.69	30.71	2.09	0.00	14.36
CIPITR, best at top beam	96.52	87.72	89.43	23.91	51.33	15.12	0.33	58.92
CIPITR, best over top 2 beams	96.55	87.78	90.48	25.85	51.72	19.85	0.41	62.52
CIPITR, best over top 5 beams	97.18	87.96	90.97	27.19	52.01	29.45	1.01	69.25
CIPITR, best over top 10 beams	97.18	88.92	90.98	28.92	52.71	32.98	1.54	73.71