Machine Translation with lexical constraints (averages over 5 runs). For each metric, we underline the top scores among all models and boldface the top scores among NAR models based on the independent student’s t-test with p < 0.05. EDITOR exploits constraints better than LevT. It also achieves comparable RIBES to the best AR model with 6–7 times decoding speedup.
| . | . | Distill . | Beam . | BLEU ↑ . | RIBES ↑ . | CPR ↑ . | Latency (ms) ↓ . |
|---|---|---|---|---|---|---|---|
| Ro-En | AR + DBA (sockeye) | 4 | 31.0 | 79.5 | 99.7 | 436.26 | |
| AR + DBA (sockeye) | 10 | 34.6 | 84.5 | 99.5 | 696.68 | ||
| NAR: LevT | ✓ | – | 31.6 | 83.4 | 80.3 | 121.80 | |
| + hard constraints | ✓ | – | 27.7 | 78.4 | 99.9 | 140.79 | |
| NAR: EDITOR | ✓ | – | 33.1 | 85.0 | 86.8 | 108.98 | |
| + hard constraints | ✓ | – | 28.8 | 81.2 | 95.0 | 136.78 | |
| En-De | AR + DBA (sockeye) | 4 | 26.1 | 74.7 | 99.7 | 434.41 | |
| AR + DBA (sockeye) | 10 | 30.5 | 81.9 | 99.5 | 896.60 | ||
| NAR: LevT | ✓ | – | 27.1 | 80.0 | 75.6 | 127.00 | |
| + hard constraints | ✓ | – | 24.9 | 74.1 | 100.0 | 134.10 | |
| NAR: EDITOR | ✓ | – | 28.2 | 81.6 | 88.4 | 121.65 | |
| + hard constraints | ✓ | – | 25.8 | 77.2 | 96.8 | 134.10 | |
| En-Ja | AR + DBA (sockeye) | 4 | 44.3 | 81.6 | 100.0 | 418.71 | |
| AR + DBA (sockeye) | 10 | 48.0 | 85.9 | 100.0 | 736.92 | ||
| NAR: LevT | ✓ | – | 42.8 | 84.0 | 74.3 | 161.17 | |
| + hard constraints | ✓ | – | 39.7 | 77.4 | 99.9 | 159.27 | |
| NAR: EDITOR | ✓ | – | 45.3 | 85.7 | 91.3 | 109.50 | |
| + hard constraints | ✓ | – | 43.7 | 82.6 | 96.4 | 132.71 | |
| . | . | Distill . | Beam . | BLEU ↑ . | RIBES ↑ . | CPR ↑ . | Latency (ms) ↓ . |
|---|---|---|---|---|---|---|---|
| Ro-En | AR + DBA (sockeye) | 4 | 31.0 | 79.5 | 99.7 | 436.26 | |
| AR + DBA (sockeye) | 10 | 34.6 | 84.5 | 99.5 | 696.68 | ||
| NAR: LevT | ✓ | – | 31.6 | 83.4 | 80.3 | 121.80 | |
| + hard constraints | ✓ | – | 27.7 | 78.4 | 99.9 | 140.79 | |
| NAR: EDITOR | ✓ | – | 33.1 | 85.0 | 86.8 | 108.98 | |
| + hard constraints | ✓ | – | 28.8 | 81.2 | 95.0 | 136.78 | |
| En-De | AR + DBA (sockeye) | 4 | 26.1 | 74.7 | 99.7 | 434.41 | |
| AR + DBA (sockeye) | 10 | 30.5 | 81.9 | 99.5 | 896.60 | ||
| NAR: LevT | ✓ | – | 27.1 | 80.0 | 75.6 | 127.00 | |
| + hard constraints | ✓ | – | 24.9 | 74.1 | 100.0 | 134.10 | |
| NAR: EDITOR | ✓ | – | 28.2 | 81.6 | 88.4 | 121.65 | |
| + hard constraints | ✓ | – | 25.8 | 77.2 | 96.8 | 134.10 | |
| En-Ja | AR + DBA (sockeye) | 4 | 44.3 | 81.6 | 100.0 | 418.71 | |
| AR + DBA (sockeye) | 10 | 48.0 | 85.9 | 100.0 | 736.92 | ||
| NAR: LevT | ✓ | – | 42.8 | 84.0 | 74.3 | 161.17 | |
| + hard constraints | ✓ | – | 39.7 | 77.4 | 99.9 | 159.27 | |
| NAR: EDITOR | ✓ | – | 45.3 | 85.7 | 91.3 | 109.50 | |
| + hard constraints | ✓ | – | 43.7 | 82.6 | 96.4 | 132.71 | |