Skip to Main Content
Table 4: 
BLEU scores for unsupervised NMT performance on WMT’14 English-French and WMT’16 English-German test sets. All methods only use unaligned Wikipedia corpora for pre-training and/or bitext mining. ‘ST’ refers to self-training.
ReferenceArchitecturePre-trainingEn-DeDe-EnEn-FrFr-En
Artetxe et al. (2018) 2-layer RNN  6.89 10.16 15.13 15.56 
Lample et al. (2018a) 3-layer RNN  9.75 13.33 15.05 14.31 
Yang et al. (2018) 4-layer Transformer  10.86 14.62 16.97 15.58 
Lample et al. (2018c) 4-layer Transformer  17.16 21.00 25.14 24.18 
Song et al. (2019) 6-layer Transformer MASS 28.3 35.2 37.5 34.9 
 
XLM Baselines 
Lample and Conneau (2019) 6-layer Transformer XLM – – 33.4 33.3 
Song et al. (2019) 6-layer Transformer XLM 27.0 34.3 33.4 33.3 
XLM reference implementation 6-layer Transformer XLM – – 36.6 34.0 
Maximum performance across baselines 6-layer Transformer XLM 27.0 34.3 36.6 34.0 
 
Ours 
Our XLM baseline 6-layer Transformer XLM 27.7 34.5 36.7 34.5 
w/ pseudo-parallel text before ST 6-layer Transformer XLM 30.4 36.3 39.7 35.9 
w/ pseudo-parallel text after ST 6-layer Transformer XLM 30.7 37.3 40.2 36.9 
ReferenceArchitecturePre-trainingEn-DeDe-EnEn-FrFr-En
Artetxe et al. (2018) 2-layer RNN  6.89 10.16 15.13 15.56 
Lample et al. (2018a) 3-layer RNN  9.75 13.33 15.05 14.31 
Yang et al. (2018) 4-layer Transformer  10.86 14.62 16.97 15.58 
Lample et al. (2018c) 4-layer Transformer  17.16 21.00 25.14 24.18 
Song et al. (2019) 6-layer Transformer MASS 28.3 35.2 37.5 34.9 
 
XLM Baselines 
Lample and Conneau (2019) 6-layer Transformer XLM – – 33.4 33.3 
Song et al. (2019) 6-layer Transformer XLM 27.0 34.3 33.4 33.3 
XLM reference implementation 6-layer Transformer XLM – – 36.6 34.0 
Maximum performance across baselines 6-layer Transformer XLM 27.0 34.3 36.6 34.0 
 
Ours 
Our XLM baseline 6-layer Transformer XLM 27.7 34.5 36.7 34.5 
w/ pseudo-parallel text before ST 6-layer Transformer XLM 30.4 36.3 39.7 35.9 
w/ pseudo-parallel text after ST 6-layer Transformer XLM 30.7 37.3 40.2 36.9 
Close Modal

or Create an Account

Close Modal
Close Modal