Skip to Main Content
Table 3: 

GLUE development set results with RoBERTa-base (top) and RoBERTa-large (bottom). We report Matthews correlation for CoLA, Pearson’s correlation for STS-B, and accuracy for others. We report mean ± standard deviation; for each bold entry, the mean minus standard deviation is no worse than RoBERTa’s corresponding mean plus standard deviation.

MNLI
ModelsCoLAMRPCRTESST-2STS-BQNLIQQPID.OOD.Avg.
RoBERTa 63.1±0.9 90.1±0.8 79.0±1.6 94.6±0.3 91.0±0.0 93.0±0.3 91.8±0.1 87.7±0.2 87.3±0.3 86.4 
 
SIFT 64.8±0.4 90.5±0.7 81.0±1.4 95.1±0.4 91.3±0.1 93.2±0.2 91.9±0.1 87.9±0.2 87.7±0.1 87.0 
SIFT-Light 64.1±1.3 90.3±0.5 80.6±1.4 94.7±0.1 91.2±0.1 92.8±0.3 91.7±0.0 87.7±0.1 87.6±0.1 86.7 
 
Syntax 63.5±0.6 90.4±0.5 80.9±1.0 94.7±0.5 91.1±0.2 92.8±0.2 91.8±0.0 87.9±0.1 87.7±0.1 86.7 
(a) Base. 
 
 MNLI  
Models CoLA MRPC RTE SST-2 STS-B QNLI QQP ID. OOD. Avg. 
RoBERTa 68.0±0.6 90.1±0.8 85.1±1.0 96.1±0.3 92.3±0.2 94.5±0.2 91.9±0.1 90.3±0.1 89.8±0.3 88.7 
 
SIFT 69.7±0.5 91.3±0.4 87.0±1.1 96.3±0.3 92.6±0.0 94.7±0.1 92.1±0.1 90.4±0.1 90.1±0.1 89.3 
Syntax 69.6±1.2 91.0±0.5 86.0±1.6 95.9±0.3 92.4±0.1 94.6±0.1 92.0±0.0 90.4±0.3 90.0±0.2 89.1 
 
(b) Large. 
MNLI
ModelsCoLAMRPCRTESST-2STS-BQNLIQQPID.OOD.Avg.
RoBERTa 63.1±0.9 90.1±0.8 79.0±1.6 94.6±0.3 91.0±0.0 93.0±0.3 91.8±0.1 87.7±0.2 87.3±0.3 86.4 
 
SIFT 64.8±0.4 90.5±0.7 81.0±1.4 95.1±0.4 91.3±0.1 93.2±0.2 91.9±0.1 87.9±0.2 87.7±0.1 87.0 
SIFT-Light 64.1±1.3 90.3±0.5 80.6±1.4 94.7±0.1 91.2±0.1 92.8±0.3 91.7±0.0 87.7±0.1 87.6±0.1 86.7 
 
Syntax 63.5±0.6 90.4±0.5 80.9±1.0 94.7±0.5 91.1±0.2 92.8±0.2 91.8±0.0 87.9±0.1 87.7±0.1 86.7 
(a) Base. 
 
 MNLI  
Models CoLA MRPC RTE SST-2 STS-B QNLI QQP ID. OOD. Avg. 
RoBERTa 68.0±0.6 90.1±0.8 85.1±1.0 96.1±0.3 92.3±0.2 94.5±0.2 91.9±0.1 90.3±0.1 89.8±0.3 88.7 
 
SIFT 69.7±0.5 91.3±0.4 87.0±1.1 96.3±0.3 92.6±0.0 94.7±0.1 92.1±0.1 90.4±0.1 90.1±0.1 89.3 
Syntax 69.6±1.2 91.0±0.5 86.0±1.6 95.9±0.3 92.4±0.1 94.6±0.1 92.0±0.0 90.4±0.3 90.0±0.2 89.1 
 
(b) Large. 
Close Modal

or Create an Account

Close Modal
Close Modal