MIT Press

Table 5

Ablation study: Performance of individual TextGen systems with the ensemble system enabled with Ranker. Here, MKT denotes MorphKey2Text, CP refers to the Compounding Module, and CR means Coreference Replacement. The symbols ‘ +’ and ‘−’ signify “with” and “without,” respectively.

		Ranker	MKTv1	MKTv2	triple2text
BLEU	− CP − CR	17.7	16.2	14.9	20.3
	+ CP − CR	30.1	29.7	27.9	30.3
	− CP + CR	29.6	29.3	28	27.8
	+ CP + CR	33.3	30.6	29.4	30.5

METEOR	− CP − CR	33.1	33.4	32.6	31.6
	+ CP − CR	38.8	39.3	38.4	36.7
	− CP + CR	37.1	37	37	34.8
	+ CP + CR	39.7	38.1	38.1	35.4

ROUGE-L	− CP − CR	50.2	51	49.1	50.6
	+ CP − CR	61.8	62.3	60.7	61
	− CP + CR	59.2	59	58.7	58.4
	+ CP + CR	64.1	63.9	62.2	62.2

STSim	− CP − CR	44.1	40.2	40.2	57.8
	+ CP − CR	85.3	85.6	85.2	83.3
	− CP + CR	82.3	82	82	79.8
	+ CP + CR	86.5	85.9	85.9	83.9

		Ranker	MKTv1	MKTv2	triple2text
BLEU	− CP − CR	17.7	16.2	14.9	20.3
	+ CP − CR	30.1	29.7	27.9	30.3
	− CP + CR	29.6	29.3	28	27.8
	+ CP + CR	33.3	30.6	29.4	30.5

METEOR	− CP − CR	33.1	33.4	32.6	31.6
	+ CP − CR	38.8	39.3	38.4	36.7
	− CP + CR	37.1	37	37	34.8
	+ CP + CR	39.7	38.1	38.1	35.4

ROUGE-L	− CP − CR	50.2	51	49.1	50.6
	+ CP − CR	61.8	62.3	60.7	61
	− CP + CR	59.2	59	58.7	58.4
	+ CP + CR	64.1	63.9	62.2	62.2

STSim	− CP − CR	44.1	40.2	40.2	57.8
	+ CP − CR	85.3	85.6	85.2	83.3
	− CP + CR	82.3	82	82	79.8
	+ CP + CR	86.5	85.9	85.9	83.9

View Large

This Feature Is Available To Subscribers Only