MIT Press

Skip to Main Content

Table 6:

bleu scores and the number of calls to score on the IWSLT’14 De-En validation set and WMT’17 De-En test set with queue size restricted to n_max ⋅ k. Note that γ = n_max is the standard best-first beam search algorithm. Performance increases are over standard beam search. Search error is with respect to beam search with same beam width.

IWSLT’14 De-En
k	γ	search error	bleu	# calls
5	2	22.7%	35.7_+0.1	43.8(163%)
	5	4.4%	35.8_+0.2	79.8(44%)
	n_max	−	35.6	93.0(24%)

10	2	22.6%	35.7_+0.3	48.4(374%)
	5	4.5%	35.6_+0.2	126.9(81%)
	n_max	−	35.4	169.0(36%)

WMT’17 De-En
5	2	29.0%	29.7_+0.2	77.5(75%)
	5	1.2%	29.5_+0.0	115.8(12%)
	n_max	−	29.5	118.8(10%)

10	2	36.6%	29.5_+0.2	97.3(165%)
	5	2.6%	29.3_+0.0	230.0(12%)
	n_max	−	29.3	230.2(12%)

IWSLT’14 De-En
k	γ	search error	bleu	# calls
5	2	22.7%	35.7_+0.1	43.8(163%)
	5	4.4%	35.8_+0.2	79.8(44%)
	n_max	−	35.6	93.0(24%)

10	2	22.6%	35.7_+0.3	48.4(374%)
	5	4.5%	35.6_+0.2	126.9(81%)
	n_max	−	35.4	169.0(36%)

WMT’17 De-En
5	2	29.0%	29.7_+0.2	77.5(75%)
	5	1.2%	29.5_+0.0	115.8(12%)
	n_max	−	29.5	118.8(10%)

10	2	36.6%	29.5_+0.2	97.3(165%)
	5	2.6%	29.3_+0.0	230.0(12%)
	n_max	−	29.3	230.2(12%)