Skip to Main Content
Table 4: 

Candidate list generation with either transformer-big or transformer-base model. The last column shows pSQM human evaluations results (higher is better). The results demonstrate that MBR needs a good model to outperform beam search.

ModelBleuBL.2pSQM ↑
Transformer-big Beam 34.3 71.6 4.47 
MBR-BL.2 25.4 79.0 4.67 
 
Transformer-base Beam 32.2 69.7 4.31 
MBR-BL.2 21.8 70.5 3.55 
 
E=base; max =big MBR-BL.2 23.5 76.2 n/a 
E=big; max =base MBR-BL.2 23.5 73.0 n/a 
ModelBleuBL.2pSQM ↑
Transformer-big Beam 34.3 71.6 4.47 
MBR-BL.2 25.4 79.0 4.67 
 
Transformer-base Beam 32.2 69.7 4.31 
MBR-BL.2 21.8 70.5 3.55 
 
E=base; max =big MBR-BL.2 23.5 76.2 n/a 
E=big; max =base MBR-BL.2 23.5 73.0 n/a 
Close Modal

or Create an Account

Close Modal
Close Modal