Skip to Main Content
Table 3: 

Translation quality from various English-based synthetic languages into standard French, using the largest training data (1.9M sentences). NMT architectures: 3-layer BiLSTM seq-to-seq with attention; 6-layer Transformer. Europarl-Test: 5K held-out Europarl sentences; Challenge set: see §5.2. All scores are averaged over three training runs.

English*→Frenchbi-lstmtransformer
Large Training (1.9M)Europarl-TestChallengeEuroparl-TestChallenge
 bleu ribes ribes bleu ribes ribes 
Original English 39.4 85.0 98.0 38.3 84.9 97.7 
 
Fixed Order: 
S-V-O 38.3 84.5 98.1 37.7 84.6 98.0 
S-O-V 37.6 84.2 97.7 37.9 84.5 97.2 
V-S-O 38.0 84.2 97.8 37.8 84.6 98.0 
V-O-S 37.8 84.0 98.0 37.6 84.3 97.2 
Average (fixed orders) 37.9±0.4 84.2±0.3 97.9±0.2 37.8±0.1 84.5±0.1 97.6±0.4 
 
Flexible Order: 
Random, no case 37.1 83.7 75.1 37.5 84.2 74.1 
Random + syncretic case 36.9 83.6 75.4 37.3 84.2 84.4 
Random + unambig. case 37.3 83.9 97.7 37.3 84.4 98.1 
 
Shuffle all words 18.5 65.2 79.4 25.8 71.2 83.2 
English*→Frenchbi-lstmtransformer
Large Training (1.9M)Europarl-TestChallengeEuroparl-TestChallenge
 bleu ribes ribes bleu ribes ribes 
Original English 39.4 85.0 98.0 38.3 84.9 97.7 
 
Fixed Order: 
S-V-O 38.3 84.5 98.1 37.7 84.6 98.0 
S-O-V 37.6 84.2 97.7 37.9 84.5 97.2 
V-S-O 38.0 84.2 97.8 37.8 84.6 98.0 
V-O-S 37.8 84.0 98.0 37.6 84.3 97.2 
Average (fixed orders) 37.9±0.4 84.2±0.3 97.9±0.2 37.8±0.1 84.5±0.1 97.6±0.4 
 
Flexible Order: 
Random, no case 37.1 83.7 75.1 37.5 84.2 74.1 
Random + syncretic case 36.9 83.6 75.4 37.3 84.2 84.4 
Random + unambig. case 37.3 83.9 97.7 37.3 84.4 98.1 
 
Shuffle all words 18.5 65.2 79.4 25.8 71.2 83.2 
Close Modal

or Create an Account

Close Modal
Close Modal