Table 2: 

Ranker performance (top-5) on dev set. Asterisk (*) indicates our best ranker used in Table 1.

IR MethodEMROUGE-L
Baseline Rankers 
BM25 18.99 47.48 
BERT DS-ranker (Mou et al., 2020) 24.26 52.68 
 - ROUGE-L filtering 22.63 51.02 
 Repl BERT w/ BiDAF 21.88 50.64 
 Repl BERT w/ MatchLSTM 21.97 50.39 
 
Our Rankers 
BERT ICT-ranker 21.29 50.35 
BERT DS-ranker 
 + Hard EM 22.45 50.50 
 + ICT pre-training* 24.83 53.19 
 
Oracle Conditions 
Upperbound (BM25 top-32) 30.81 61.40 
Oracle (BM25 w/ Q+A) 35.75 63.92 
IR MethodEMROUGE-L
Baseline Rankers 
BM25 18.99 47.48 
BERT DS-ranker (Mou et al., 2020) 24.26 52.68 
 - ROUGE-L filtering 22.63 51.02 
 Repl BERT w/ BiDAF 21.88 50.64 
 Repl BERT w/ MatchLSTM 21.97 50.39 
 
Our Rankers 
BERT ICT-ranker 21.29 50.35 
BERT DS-ranker 
 + Hard EM 22.45 50.50 
 + ICT pre-training* 24.83 53.19 
 
Oracle Conditions 
Upperbound (BM25 top-32) 30.81 61.40 
Oracle (BM25 w/ Q+A) 35.75 63.92 
View Large
Close Modal
This Feature Is Available To Subscribers Only

or Create an Account

Close Modal
Close Modal
This site uses cookies. By continuing to use our website, you are agreeing to our privacy policy. No content on this site may be used to train artificial intelligence systems without permission in writing from the MIT Press.
Accept