Skip to Main Content
Table 1: 
Number of sentences per set and subcorpus. For each subcorpus, 𝒜train includes 𝒜in_probe and does not include 𝒜out_probe. ℬall is a subset of 𝒜train, excluding 𝒜in_probe and ParaCrawl. 𝒜ood is for evaluation only, and only Carol has access to it.
𝒜out_probe𝒜in_probe𝒜trainall𝒜ood
ParaCrawl 5,000 5,000 4,518,029 N/A 
CommonCrawl 5,000 5,000 2,389,123 2,379,123 N/A 
Europarl 5,000 5,000 1,865,271 1,855,271 N/A 
News 5,000 5,000 273,702 263,702 N/A 
Rapid 5,000 5,000 1,062,214 1,052,214 N/A 
 
EMEA N/A N/A N/A N/A 5,000 
Koran N/A N/A N/A N/A 5,000 
Subtitles N/A N/A N/A N/A 5,000 
TED N/A N/A N/A N/A 5,000 
 
TOTAL 25,000 25,000 10,108,339 5,550,310 20,000 
𝒜out_probe𝒜in_probe𝒜trainall𝒜ood
ParaCrawl 5,000 5,000 4,518,029 N/A 
CommonCrawl 5,000 5,000 2,389,123 2,379,123 N/A 
Europarl 5,000 5,000 1,865,271 1,855,271 N/A 
News 5,000 5,000 273,702 263,702 N/A 
Rapid 5,000 5,000 1,062,214 1,052,214 N/A 
 
EMEA N/A N/A N/A N/A 5,000 
Koran N/A N/A N/A N/A 5,000 
Subtitles N/A N/A N/A N/A 5,000 
TED N/A N/A N/A N/A 5,000 
 
TOTAL 25,000 25,000 10,108,339 5,550,310 20,000 
Close Modal

or Create an Account

Close Modal
Close Modal