Results of abstractiveness-controlled models with different specified bins on Newsroom-b dataset. Bin 3 is the most abstractive bin. Our CMDP framework significantly improves the BERTScore, MoverScore, and bin % over all the bins (p < 0.003, approx. randomization test).
| . | bin 1 . | bin 2 . | bin 3 . | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Method . | BERTS. . | MoverS. . | Bin % . | BERTS. . | MoverS. . | Bin % . | BERTS. . | MoverS. . | Bin % . |
| ControlSum | 26.53 | 16.15 | 99.40 | 24.50 | 13.11 | 6.42 | 20.53 | 9.87 | 24.30 |
| PG | 26.49 | 15.99 | 99.36 | 22.59 | 11.15 | 8.49 | 17.67 | 7.45 | 26.03 |
| PG+CMDP | 29.62 | 18.44 | 99.72 | 26.95 | 12.96 | 97.65 | 22.78 | 7.67 | 98.88 |
| D.GPT2 | 27.85 | 17.05 | 99.17 | 26.41 | 14.57 | 0.47 | 22.30 | 11.16 | 00.37 |
| D.GPT2+CMDP | 30.12 | 18.45 | 99.72 | 31.21 | 17.52 | 72.48 | 25.75 | 13.18 | 87.09 |
| D.GPT2+MDP | 29.77 | 18.27 | 99.77 | 30.40 | 17.06 | 72.37 | 25.25 | 13.06 | 80.95 |
| . | bin 1 . | bin 2 . | bin 3 . | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Method . | BERTS. . | MoverS. . | Bin % . | BERTS. . | MoverS. . | Bin % . | BERTS. . | MoverS. . | Bin % . |
| ControlSum | 26.53 | 16.15 | 99.40 | 24.50 | 13.11 | 6.42 | 20.53 | 9.87 | 24.30 |
| PG | 26.49 | 15.99 | 99.36 | 22.59 | 11.15 | 8.49 | 17.67 | 7.45 | 26.03 |
| PG+CMDP | 29.62 | 18.44 | 99.72 | 26.95 | 12.96 | 97.65 | 22.78 | 7.67 | 98.88 |
| D.GPT2 | 27.85 | 17.05 | 99.17 | 26.41 | 14.57 | 0.47 | 22.30 | 11.16 | 00.37 |
| D.GPT2+CMDP | 30.12 | 18.45 | 99.72 | 31.21 | 17.52 | 72.48 | 25.75 | 13.18 | 87.09 |
| D.GPT2+MDP | 29.77 | 18.27 | 99.77 | 30.40 | 17.06 | 72.37 | 25.25 | 13.06 | 80.95 |