Skip to Main Content
Table 7: 
The effects of different auxiliary objectives, given MLM over random spans as the primary objective.
SQuAD 2.0NewsQATriviaQACorefMNLI-mQNLIGLUE (Avg)
Span Masking (2seq) + NSP 85.4 73.0 78.8 76.4 87.0 93.3 83.4 
Span Masking (1seq) 86.7 73.4 80.0 76.3 87.3 93.8 83.8 
Span Masking (1seq) + SBO 86.8 74.1 80.3 79.0 87.6 93.9 84.0 
SQuAD 2.0NewsQATriviaQACorefMNLI-mQNLIGLUE (Avg)
Span Masking (2seq) + NSP 85.4 73.0 78.8 76.4 87.0 93.3 83.4 
Span Masking (1seq) 86.7 73.4 80.0 76.3 87.3 93.8 83.8 
Span Masking (1seq) + SBO 86.8 74.1 80.3 79.0 87.6 93.9 84.0 
View Large
Close Modal
This Feature Is Available To Subscribers Only

or Create an Account

Close Modal
Close Modal
This site uses cookies. By continuing to use our website, you are agreeing to our privacy policy.
Accept