Summarization results on Yelp and Amazon. Best system, shown in boldface, is significantly better than all comparison systems, except where underlined (p < 0.05; paired bootstrap resampling; Koehn, 2004).
| Yelp . | R1 . | R2 . | RL . | ACF1 . | SCMSE . |
|---|---|---|---|---|---|
| Random | 23.04 | 2.44 | 13.44 | .551 | .612 |
| CentroidBERT | 24.78 | 2.64 | 14.67 | .691 | .523 |
| OracleBERT | 27.38 | 3.75 | 15.92 | .703 | .507 |
| LexRankBERT | 26.46 | 3.00 | 14.36 | .601 | .541 |
| Opinosis | 24.88 | 2.78 | 14.09 | .672 | .552 |
| MeanSum | 28.46 | 3.66 | 15.57 | .713 | .510 |
| Copycat | 29.47 | 5.26 | 18.09 | .728 | .495 |
| QT | 28.40 | 3.97 | 15.27 | .722 | .490 |
| Amazon | R1 | R2 | RL | ACF1 | SCMSE |
| Random | 27.66 | 4.72 | 16.95 | .580 | .602 |
| CentroidBERT | 29.94 | 5.19 | 17.70 | .702 | .599 |
| OracleBERT | 31.69 | 6.47 | 19.25 | .725 | .512 |
| LexRankBERT | 31.47 | 5.07 | 16.81 | .663 | .541 |
| Opinosis | 28.42 | 4.57 | 15.50 | .614 | .580 |
| MeanSum | 29.20 | 4.70 | 18.15 | .710 | .525 |
| CopyCat | 31.97 | 5.81 | 20.16 | .731 | .510 |
| QT | 34.04 | 7.03 | 18.08 | .739 | .508 |
| Yelp . | R1 . | R2 . | RL . | ACF1 . | SCMSE . |
|---|---|---|---|---|---|
| Random | 23.04 | 2.44 | 13.44 | .551 | .612 |
| CentroidBERT | 24.78 | 2.64 | 14.67 | .691 | .523 |
| OracleBERT | 27.38 | 3.75 | 15.92 | .703 | .507 |
| LexRankBERT | 26.46 | 3.00 | 14.36 | .601 | .541 |
| Opinosis | 24.88 | 2.78 | 14.09 | .672 | .552 |
| MeanSum | 28.46 | 3.66 | 15.57 | .713 | .510 |
| Copycat | 29.47 | 5.26 | 18.09 | .728 | .495 |
| QT | 28.40 | 3.97 | 15.27 | .722 | .490 |
| Amazon | R1 | R2 | RL | ACF1 | SCMSE |
| Random | 27.66 | 4.72 | 16.95 | .580 | .602 |
| CentroidBERT | 29.94 | 5.19 | 17.70 | .702 | .599 |
| OracleBERT | 31.69 | 6.47 | 19.25 | .725 | .512 |
| LexRankBERT | 31.47 | 5.07 | 16.81 | .663 | .541 |
| Opinosis | 28.42 | 4.57 | 15.50 | .614 | .580 |
| MeanSum | 29.20 | 4.70 | 18.15 | .710 | .525 |
| CopyCat | 31.97 | 5.81 | 20.16 | .731 | .510 |
| QT | 34.04 | 7.03 | 18.08 | .739 | .508 |