Skip to Main Content
Table 2: 
Results on QQP and ParaNMT-small dataset. Higher↑ BLEU, METEOR, ROUGE, and PDS is better whereas lower↓ TED score is better. Sgcp-R selects the best candidate out of many, resulting in performance boost for semantic preservation (shown in box). We bold the statistically significant results of Sgcp-F, only, for a fair comparison with the baselines. Note that Source-as-Output, and Exemplar-as-Output are only dataset quality indicators and not the competitive baselines. Please see Section 5 for details.
QQP-Pos
Model BLEU ↑ METEOR ↑ ROUGE-1 ↑ ROUGE-2 ↑ ROUGE-L ↑ TED-R TED-E PDS ↑ 
Source-as-Output 17.2 31.1 51.9 26.2 52.9 16.2 16.6 99.8 
Exemplar-as-Output 16.8 17.6 38.2 20.5 43.2 4.8 0.0 10.7 
Scpn (Iyyer et al., 2018) 15.6 19.6 40.6 20.5 44.6 9.1 8.0 27.0 
Cgen (Chen et al., 2019a) 34.9 37.4 62.6 42.7 65.4 6.7 6.0 65.4 
Sgcp-F 36.7 39.8 66.9 45.0 69.6 4.8 1.8 75.0 
Sgcp-R 38.0 41.3 68.1 45.7 70.2 6.8 5.9 87.7 
 
ParaNMT-small 
Source-as-Output 18.5 28.8 50.6 23.1 47.7 12.0 13.0 99.0 
Exemplar-as-Output 3.3 12.1 24.4 7.5 29.1 5.9 0.0 14.0 
Scpn (Iyyer et al., 2018) 6.4 14.6 30.3 11.2 34.6 6.2 1.4 15.4 
Cgen (Chen et al., 2019a) 13.6 24.8 44.8 21.0 48.3 6.7 3.3 70.2 
Sgcp-F 15.3 25.9 46.6 21.8 49.7 6.1 1.4 76.6 
Sgcp-R 16.4 27.2 49.6 22.9 50.5 8.7 7.0 83.5 
QQP-Pos
Model BLEU ↑ METEOR ↑ ROUGE-1 ↑ ROUGE-2 ↑ ROUGE-L ↑ TED-R TED-E PDS ↑ 
Source-as-Output 17.2 31.1 51.9 26.2 52.9 16.2 16.6 99.8 
Exemplar-as-Output 16.8 17.6 38.2 20.5 43.2 4.8 0.0 10.7 
Scpn (Iyyer et al., 2018) 15.6 19.6 40.6 20.5 44.6 9.1 8.0 27.0 
Cgen (Chen et al., 2019a) 34.9 37.4 62.6 42.7 65.4 6.7 6.0 65.4 
Sgcp-F 36.7 39.8 66.9 45.0 69.6 4.8 1.8 75.0 
Sgcp-R 38.0 41.3 68.1 45.7 70.2 6.8 5.9 87.7 
 
ParaNMT-small 
Source-as-Output 18.5 28.8 50.6 23.1 47.7 12.0 13.0 99.0 
Exemplar-as-Output 3.3 12.1 24.4 7.5 29.1 5.9 0.0 14.0 
Scpn (Iyyer et al., 2018) 6.4 14.6 30.3 11.2 34.6 6.2 1.4 15.4 
Cgen (Chen et al., 2019a) 13.6 24.8 44.8 21.0 48.3 6.7 3.3 70.2 
Sgcp-F 15.3 25.9 46.6 21.8 49.7 6.1 1.4 76.6 
Sgcp-R 16.4 27.2 49.6 22.9 50.5 8.7 7.0 83.5 
Close Modal

or Create an Account

Close Modal
Close Modal