Skip to Main Content
Table 10: 

Human fluency, relevance, and faithfulness scores of abstractiveness-controlled models on Newsroom-b. The Krippendorf’s α inter-rater agreement for these metrics are 0.51, 0.37, and 0.40.

BinMethodFlu.Rel.Faithful.
PG+CMDP 4.79 3.43 98% 
D.GPT2+CMDP 4.75 3.34 96% 
PG+CMDP 4.52 2.34 58% 
D.GPT2+CMDP 4.57 3.14 66% 
PG+CMDP 4.47 2.00 52% 
D.GPT2+CMDP 4.60 2.99 66% 
BinMethodFlu.Rel.Faithful.
PG+CMDP 4.79 3.43 98% 
D.GPT2+CMDP 4.75 3.34 96% 
PG+CMDP 4.52 2.34 58% 
D.GPT2+CMDP 4.57 3.14 66% 
PG+CMDP 4.47 2.00 52% 
D.GPT2+CMDP 4.60 2.99 66% 
Close Modal

or Create an Account

Close Modal
Close Modal