Skip to Main Content
Table 3: 

CamRest676 test results (end-to-end modeling with generated beliefs) with seq2seq approaches. Noisy channel reranking performs comparable with noisy channel online decoding, and the results are not shown. Results are significant (p ¡ 0.01) comparing noisy channel decoding and direct decoding.

ModelInformSuccessBLEUCombined
Sequicity (Lei et al., 2018) 92.3 85.3 21.40 110.20 
GPT-2 fine-tuned (Wu et al., 2019b) – 86.2 19.20 – 
ARDM (Wu et al., 2019b) – 87.1 25.20 – 
SOLOIST (Peng et al., 2020a) 94.7 87.1 25.50 116.40 
 
Randomly Initialized 
Direct decoding (114M) 78.1 83.5 21.58 102.38 
Noisy channel online decoding (116M) 79.8 84.1 22.83 104.78 
Noisy channel online decoding (292M) 80.9 84.9 23.19 106.09 
 
Reddit Pretraining 
Direct decoding (114M) 93.3 83.9 23.41 112.01 
Noisy channel online decoding (116M) 93.7 84.5 25.14 114.24 
Noisy channel online decoding (292M) 93.9 84.7 25.38 114.68 
 
Task-Oriented Pretraining 
Direct decoding (114M) 93.4 84.3 24.92 113.77 
Noisy channel online decoding (116M) 94.3 85.2 25.98 115.73 
Noisy channel online decoding (292M) 95.4 85.3 26.89 117.24 
ModelInformSuccessBLEUCombined
Sequicity (Lei et al., 2018) 92.3 85.3 21.40 110.20 
GPT-2 fine-tuned (Wu et al., 2019b) – 86.2 19.20 – 
ARDM (Wu et al., 2019b) – 87.1 25.20 – 
SOLOIST (Peng et al., 2020a) 94.7 87.1 25.50 116.40 
 
Randomly Initialized 
Direct decoding (114M) 78.1 83.5 21.58 102.38 
Noisy channel online decoding (116M) 79.8 84.1 22.83 104.78 
Noisy channel online decoding (292M) 80.9 84.9 23.19 106.09 
 
Reddit Pretraining 
Direct decoding (114M) 93.3 83.9 23.41 112.01 
Noisy channel online decoding (116M) 93.7 84.5 25.14 114.24 
Noisy channel online decoding (292M) 93.9 84.7 25.38 114.68 
 
Task-Oriented Pretraining 
Direct decoding (114M) 93.4 84.3 24.92 113.77 
Noisy channel online decoding (116M) 94.3 85.2 25.98 115.73 
Noisy channel online decoding (292M) 95.4 85.3 26.89 117.24 
Close Modal

or Create an Account

Close Modal
Close Modal