Mean BLEU scores and exact match accuracies on the three MCD splits and on a random split in zero-shot cross-lingual transfer experiments on MCWQ. The gray text represents the models’ monolingual performance on English, given for reference (the exact match accuracies are copied from Table 4). The black text indicates the zero-shot cross-lingual transfer performances on Hebrew, Kannada, and Chinese of a model trained on English. While the scores for individual MCD splits are omitted for brevity, in all three MCD splits, the accuracies are below 1% (except on MCD2 Chinese, being 4%).
SPARQL BLEU . | MCDmean . | Random . | ||||||
---|---|---|---|---|---|---|---|---|
En . | He . | Kn . | Zh . | En . | He . | Kn . | Zh . | |
mT5-small+RIR | 87.5 | 53.8 | 53.2 | 59 | 99.9 | 60.4 | 59.9 | 63.8 |
mT5-base+RIR | 86.4 | 46.4 | 46 | 52.7 | 99.9 | 63.2 | 63.5 | 70.6 |
Exact Match (%) | ||||||||
mT5-small+RIR | 38.3 | 0.2 | 0.3 | 0.2 | 98.6 | 0.5 | 0.4 | 1.1 |
mT5-base+RIR | 33.8 | 0.4 | 0.7 | 1.5 | 99.1 | 1.1 | 0.9 | 7.2 |
SPARQL BLEU . | MCDmean . | Random . | ||||||
---|---|---|---|---|---|---|---|---|
En . | He . | Kn . | Zh . | En . | He . | Kn . | Zh . | |
mT5-small+RIR | 87.5 | 53.8 | 53.2 | 59 | 99.9 | 60.4 | 59.9 | 63.8 |
mT5-base+RIR | 86.4 | 46.4 | 46 | 52.7 | 99.9 | 63.2 | 63.5 | 70.6 |
Exact Match (%) | ||||||||
mT5-small+RIR | 38.3 | 0.2 | 0.3 | 0.2 | 98.6 | 0.5 | 0.4 | 1.1 |
mT5-base+RIR | 33.8 | 0.4 | 0.7 | 1.5 | 99.1 | 1.1 | 0.9 | 7.2 |