Experimental results over Wikipedia and CNN, in both in-domain and cross-domain settings. Acc is at the document level and F1 is at the sentence level.
| . | → Wikipedia . | → CNN . | ||||||
|---|---|---|---|---|---|---|---|---|
| . | Wiki→Wiki . | CNN→Wiki . | CNN→CNN . | Wiki→CNN . | ||||
| . | Acc (%) . | F1 (%) . | Acc (%) . | F1 (%) . | Acc (%) . | F1 (%) . | Acc (%) . | F1 (%) . |
| Majority-class | 57.3 | 0.0 | 57.3 | 0.0 | 50.6 | 0.0 | 50.6 | 0.0 |
| BoW | 57.3 | 0.0 | 57.3 | 0.0 | 50.6 | 0.0 | 50.6 | 0.0 |
| Bi-LSTM | 56.2 | 12.7 | 57.3 | 0.0 | 51.7 | 25.1 | 50.2 | 3.0 |
| InferSent | 57.3 | 0.0 | 57.3 | 0.0 | 50.6 | 0.0 | 50.6 | 0.0 |
| Skip-Thought | 57.3 | 0.0 | 57.3 | 0.0 | 50.6 | 0.0 | 50.6 | 0.0 |
| BERT-Base | 65.3 | 35.7 | 61.2 | 21.1 | 80.8 | 71.6 | 57.0 | 23.5 |
| BERT-Large | 67.0 | 39.6 | 64.0 | 29.1 | 82.4 | 74.8 | 61.5 | 35.9 |
| XLNet-Base | 67.8 | 45.0 | 62.2 | 22.4 | 91.2 | 86.6 | 64.0 | 43.3 |
| XLNet-Large | 72.9 | 55.4 | 62.8 | 22.2 | 96.9 | 95.0 | 80.7 | 73.8 |
| RoBERTa-Base | 69.5 | 47.0 | 63.2 | 26.1 | 92.5 | 88.8 | 77.6 | 68.1 |
| RoBERTa-Large | 76.1 | 59.8 | 63.7 | 24.6 | 96.0 | 94.5 | 88.3 | 83.5 |
| ALBERT-Large | 70.7 | 49.6 | 63.8 | 24.9 | 93.4 | 90.8 | 72.6 | 61.5 |
| ALBERT-xxLarge | 81.7 | 71.5 | 66.6 | 33.2 | 96.9 | 95.9 | 89.1 | 86.7 |
| ALBERT-xxLarge-freeze | 57.3 | 0.0 | N/A | N/A | 50.6 | 0.3 | N/A | N/A |
| Human | 66.6 | 35.9 | 66.6 | 35.9 | 74.0 | 57.8 | ||
| . | → Wikipedia . | → CNN . | ||||||
|---|---|---|---|---|---|---|---|---|
| . | Wiki→Wiki . | CNN→Wiki . | CNN→CNN . | Wiki→CNN . | ||||
| . | Acc (%) . | F1 (%) . | Acc (%) . | F1 (%) . | Acc (%) . | F1 (%) . | Acc (%) . | F1 (%) . |
| Majority-class | 57.3 | 0.0 | 57.3 | 0.0 | 50.6 | 0.0 | 50.6 | 0.0 |
| BoW | 57.3 | 0.0 | 57.3 | 0.0 | 50.6 | 0.0 | 50.6 | 0.0 |
| Bi-LSTM | 56.2 | 12.7 | 57.3 | 0.0 | 51.7 | 25.1 | 50.2 | 3.0 |
| InferSent | 57.3 | 0.0 | 57.3 | 0.0 | 50.6 | 0.0 | 50.6 | 0.0 |
| Skip-Thought | 57.3 | 0.0 | 57.3 | 0.0 | 50.6 | 0.0 | 50.6 | 0.0 |
| BERT-Base | 65.3 | 35.7 | 61.2 | 21.1 | 80.8 | 71.6 | 57.0 | 23.5 |
| BERT-Large | 67.0 | 39.6 | 64.0 | 29.1 | 82.4 | 74.8 | 61.5 | 35.9 |
| XLNet-Base | 67.8 | 45.0 | 62.2 | 22.4 | 91.2 | 86.6 | 64.0 | 43.3 |
| XLNet-Large | 72.9 | 55.4 | 62.8 | 22.2 | 96.9 | 95.0 | 80.7 | 73.8 |
| RoBERTa-Base | 69.5 | 47.0 | 63.2 | 26.1 | 92.5 | 88.8 | 77.6 | 68.1 |
| RoBERTa-Large | 76.1 | 59.8 | 63.7 | 24.6 | 96.0 | 94.5 | 88.3 | 83.5 |
| ALBERT-Large | 70.7 | 49.6 | 63.8 | 24.9 | 93.4 | 90.8 | 72.6 | 61.5 |
| ALBERT-xxLarge | 81.7 | 71.5 | 66.6 | 33.2 | 96.9 | 95.9 | 89.1 | 86.7 |
| ALBERT-xxLarge-freeze | 57.3 | 0.0 | N/A | N/A | 50.6 | 0.3 | N/A | N/A |
| Human | 66.6 | 35.9 | 66.6 | 35.9 | 74.0 | 57.8 | ||