Skip to Main Content
Table 10: 

Case study on the responses decoded by direct decoding, noisy channel reranking, and noisy channel online decoding. The large noisy channel model is used.

ModelDelexicalized ResponseSacreBLEU
Dialogue: MUL1624, Turn Number: 4 
Ground truth Sure. The trainid is [train_id]. It leaves at [value_time] and arrives at [value_time]. Would you like to make a reservation? – 
Direct decoding The travel time is [value_count] minutes. 2.03 
Reranking [train_id] leaves [value_place] at [value_time] and arrives in [value_place] at [value_time]. Would you like me to book that for you? 46.33 
Online decoding [train_id] leaves at [value_time] and arrives at [value_time]. Would you like me to book that? 60.90 
 
Dialogue: MUL1276, Turn Number: 6 
Ground truth The [restaurant_name] has a postal code of [restaurant_postcode]. Will this work for you? I can book it if you please. – 
Direct decoding There are [value_count] places. What type of food would you like? 1.63 
Reranking How about [restaurant_name] located at [restaurant_address], postcode [restaurant_postcode]. 24.90 
Online decoding How about [restaurant_name] located at [restaurant_address], postcode [restaurant_postcode]? 22.00 
 
Dialogue: MUL1898, Turn Number: 2 
Ground truth Yes, [attraction_name] is on [attraction_address] and is in the [value_area] side of town. Is there anything else you need to know? – 
Direct decoding [attraction_name] is located in the [value_area] part of town and has free admission. 27.53 
Reranking [attraction_name] is located in the [value_area] of town at [attraction_address]. The entrance fee is free. Can I help you with anything else? 41.66 
Online decoding [attraction_name] is located in the [value_area] part of town at [attraction_address]. Can I help you with anything else? 42.38 
ModelDelexicalized ResponseSacreBLEU
Dialogue: MUL1624, Turn Number: 4 
Ground truth Sure. The trainid is [train_id]. It leaves at [value_time] and arrives at [value_time]. Would you like to make a reservation? – 
Direct decoding The travel time is [value_count] minutes. 2.03 
Reranking [train_id] leaves [value_place] at [value_time] and arrives in [value_place] at [value_time]. Would you like me to book that for you? 46.33 
Online decoding [train_id] leaves at [value_time] and arrives at [value_time]. Would you like me to book that? 60.90 
 
Dialogue: MUL1276, Turn Number: 6 
Ground truth The [restaurant_name] has a postal code of [restaurant_postcode]. Will this work for you? I can book it if you please. – 
Direct decoding There are [value_count] places. What type of food would you like? 1.63 
Reranking How about [restaurant_name] located at [restaurant_address], postcode [restaurant_postcode]. 24.90 
Online decoding How about [restaurant_name] located at [restaurant_address], postcode [restaurant_postcode]? 22.00 
 
Dialogue: MUL1898, Turn Number: 2 
Ground truth Yes, [attraction_name] is on [attraction_address] and is in the [value_area] side of town. Is there anything else you need to know? – 
Direct decoding [attraction_name] is located in the [value_area] part of town and has free admission. 27.53 
Reranking [attraction_name] is located in the [value_area] of town at [attraction_address]. The entrance fee is free. Can I help you with anything else? 41.66 
Online decoding [attraction_name] is located in the [value_area] part of town at [attraction_address]. Can I help you with anything else? 42.38 
Close Modal

or Create an Account

Close Modal
Close Modal