MIT Press

Table 1:

Number of sentences per set and subcorpus. For each subcorpus,

𝒜_{t r a i n}

includes

𝒜_{i n_p r o b e}

and does not include

𝒜_{o u t_p r o b e}

⁠. ℬ_all is a subset of

𝒜_{t r a i n}

⁠, excluding

𝒜_{i n_p r o b e}

and ParaCrawl.

𝒜_{o o d}

is for evaluation only, and only Carol has access to it.

	$𝒜_{o u t_p r o b e}$	$𝒜_{i n_p r o b e}$	$𝒜_{t r a i n}$	ℬ_all	$𝒜_{o o d}$
ParaCrawl	5,000	5,000	4,518,029	0	N/A
CommonCrawl	5,000	5,000	2,389,123	2,379,123	N/A
Europarl	5,000	5,000	1,865,271	1,855,271	N/A
News	5,000	5,000	273,702	263,702	N/A
Rapid	5,000	5,000	1,062,214	1,052,214	N/A

EMEA	N/A	N/A	N/A	N/A	5,000
Koran	N/A	N/A	N/A	N/A	5,000
Subtitles	N/A	N/A	N/A	N/A	5,000
TED	N/A	N/A	N/A	N/A	5,000

TOTAL	25,000	25,000	10,108,339	5,550,310	20,000

	$𝒜_{o u t_p r o b e}$	$𝒜_{i n_p r o b e}$	$𝒜_{t r a i n}$	ℬ_all	$𝒜_{o o d}$
ParaCrawl	5,000	5,000	4,518,029	0	N/A
CommonCrawl	5,000	5,000	2,389,123	2,379,123	N/A
Europarl	5,000	5,000	1,865,271	1,855,271	N/A
News	5,000	5,000	273,702	263,702	N/A
Rapid	5,000	5,000	1,062,214	1,052,214	N/A

EMEA	N/A	N/A	N/A	N/A	5,000
Koran	N/A	N/A	N/A	N/A	5,000
Subtitles	N/A	N/A	N/A	N/A	5,000
TED	N/A	N/A	N/A	N/A	5,000

TOTAL	25,000	25,000	10,108,339	5,550,310	20,000