Table 5: 

Mean BLEU scores and exact match accuracies on the three MCD splits and on a random split in zero-shot cross-lingual transfer experiments on MCWQ. The gray text represents the models’ monolingual performance on English, given for reference (the exact match accuracies are copied from Table 4). The black text indicates the zero-shot cross-lingual transfer performances on Hebrew, Kannada, and Chinese of a model trained on English. While the scores for individual MCD splits are omitted for brevity, in all three MCD splits, the accuracies are below 1% (except on MCD2 Chinese, being 4%).

SPARQL BLEUMCDmeanRandom
EnHeKnZhEnHeKnZh
mT5-small+RIR 87.5 53.8 53.2 59 99.9 60.4 59.9 63.8 
mT5-base+RIR 86.4 46.4 46 52.7 99.9 63.2 63.5 70.6 
 
Exact Match (%) 
mT5-small+RIR 38.3 0.2 0.3 0.2 98.6 0.5 0.4 1.1 
mT5-base+RIR 33.8 0.4 0.7 1.5 99.1 1.1 0.9 7.2 
SPARQL BLEUMCDmeanRandom
EnHeKnZhEnHeKnZh
mT5-small+RIR 87.5 53.8 53.2 59 99.9 60.4 59.9 63.8 
mT5-base+RIR 86.4 46.4 46 52.7 99.9 63.2 63.5 70.6 
 
Exact Match (%) 
mT5-small+RIR 38.3 0.2 0.3 0.2 98.6 0.5 0.4 1.1 
mT5-base+RIR 33.8 0.4 0.7 1.5 99.1 1.1 0.9 7.2 
Close Modal

or Create an Account

Close Modal
Close Modal