Skip to Main Content
Table A.1: 

Results on ICT task and NQ task (correspond to Figure 4 and Figure 5).

ModelRerankingRetrieval
Passage length 50 100 200 400 50 100 200 400 
 
ICT task (MRR@10) 
Cross-Attention 99.9 99.9 99.8 99.6 
HYBRID-ME-BERT-uni 98.2 97.0 94.4 91.9 
HYBRID-ME-BERT-bi 99.3 99.0 97.3 96.1 
ME-BERT-768 98.0 96.7 92.4 89.8 96.8 96.1 91.1 85.2 
ME-BERT-64 96.3 94.2 89.0 83.7 92.9 91.7 84.6 72.8 
DE-BERT-768 91.7 87.8 79.7 74.1 90.2 85.6 72.9 63.0 
DE-BERT-512 91.4 87.2 78.9 73.1 89.4 81.5 66.8 55.8 
DE-BERT-128 90.5 85.0 75.0 68.1 85.7 75.4 58.0 47.3 
DE-BERT-64 88.8 82.0 70.7 63.8 82.8 68.9 48.5 38.3 
DE-BERT-32 83.6 74.9 62.6 55.9 70.1 53.2 34.0 27.6 
BM25-uni 92.1 88.6 84.6 81.8 92.1 88.6 84.6 81.8 
BM25-bi 98.0 97.1 95.9 94.5 98.0 97.1 95.9 94.5 
 
NQ (Recall@400 tokens) 
Cross-Attention 48.9 55.5 54.2 47.6 
HYBRID-ME-BERT-uni 45.7 49.5 48.5 42.9 
ME-BERT-768 43.6 49.6 46.5 38.7 42.0 43.3 40.4 34.4 
ME-BERT-64 44.4 48.7 44.5 38.2 42.2 43.4 38.9 33.0 
DE-BERT-768 42.9 47.7 44.4 36.6 44.2 44.0 40.1 32.2 
DE-BERT-512 43.8 48.5 44.1 36.5 43.3 43.2 38.8 32.7 
DE-BERT-128 42.8 45.7 41.2 35.7 38.0 36.7 32.8 27.0 
DE-BERT-64 42.6 45.7 42.5 35.4 37.4 35.1 32.6 26.6 
DE-BERT-32 42.4 45.8 42.1 34.0 36.3 34.7 31.0 24.9 
BM25-uni 30.1 35.7 34.1 30.1 30.1 35.7 34.1 30.1 
ModelRerankingRetrieval
Passage length 50 100 200 400 50 100 200 400 
 
ICT task (MRR@10) 
Cross-Attention 99.9 99.9 99.8 99.6 
HYBRID-ME-BERT-uni 98.2 97.0 94.4 91.9 
HYBRID-ME-BERT-bi 99.3 99.0 97.3 96.1 
ME-BERT-768 98.0 96.7 92.4 89.8 96.8 96.1 91.1 85.2 
ME-BERT-64 96.3 94.2 89.0 83.7 92.9 91.7 84.6 72.8 
DE-BERT-768 91.7 87.8 79.7 74.1 90.2 85.6 72.9 63.0 
DE-BERT-512 91.4 87.2 78.9 73.1 89.4 81.5 66.8 55.8 
DE-BERT-128 90.5 85.0 75.0 68.1 85.7 75.4 58.0 47.3 
DE-BERT-64 88.8 82.0 70.7 63.8 82.8 68.9 48.5 38.3 
DE-BERT-32 83.6 74.9 62.6 55.9 70.1 53.2 34.0 27.6 
BM25-uni 92.1 88.6 84.6 81.8 92.1 88.6 84.6 81.8 
BM25-bi 98.0 97.1 95.9 94.5 98.0 97.1 95.9 94.5 
 
NQ (Recall@400 tokens) 
Cross-Attention 48.9 55.5 54.2 47.6 
HYBRID-ME-BERT-uni 45.7 49.5 48.5 42.9 
ME-BERT-768 43.6 49.6 46.5 38.7 42.0 43.3 40.4 34.4 
ME-BERT-64 44.4 48.7 44.5 38.2 42.2 43.4 38.9 33.0 
DE-BERT-768 42.9 47.7 44.4 36.6 44.2 44.0 40.1 32.2 
DE-BERT-512 43.8 48.5 44.1 36.5 43.3 43.2 38.8 32.7 
DE-BERT-128 42.8 45.7 41.2 35.7 38.0 36.7 32.8 27.0 
DE-BERT-64 42.6 45.7 42.5 35.4 37.4 35.1 32.6 26.6 
DE-BERT-32 42.4 45.8 42.1 34.0 36.3 34.7 31.0 24.9 
BM25-uni 30.1 35.7 34.1 30.1 30.1 35.7 34.1 30.1 
Close Modal

or Create an Account

Close Modal
Close Modal