Skip to Main Content
Table 7

SRL results with different pre-trained language models on CoNLL-2005, CoNLL-2009, and CoNLL-2012 test sets. The results listed in the table are evaluated while given pre-identified predicates. The baseline is our proposed graph-based model because of its good performance. The “SYN” column indicates whether syntax information is employed, and the GCN syntax encoder is adopted for all models that are enhanced with syntax information. In this table, “△” in brackets represents relative improvements using syntax information compared to the syntax-agnostic model, while the “” indicates an absolute improvement using a pre-trained language model compared to the pure baseline model without any syntax information or pre-trained language model enhancement.

SystemSYNCoNLL09 WSJCoNLL09 BrownCoNLL05 WSJCoNLL05 BrownCoNLL12
Baseline 87.8 79.2 84.5 74.8 83.5 
89.2 (△ +1.4) 80.1 (△ +0.9) 85.6 (△ +1.1) 76.2 (△ +1.4) 84.7 (△ +1.2) 
ELMo 90.4 ( +2.6) 81.5 ( +2.3) 87.7 ( +3.2) 80.5 ( +5.7) 86.0 ( +2.5) 
91.1 (△ +0.7) 82.1 (△ +0.6) 88.6 (△ +0.9) 81.0 (△ +0.5) 86.7 (△ +0.7) 
BERT 91.4 ( +3.6) 82.8 ( +3.6) 89.0 ( +4.5) 82.3 ( +7.5) 87.4 ( +3.9) 
91.8 (△ +0.4) 83.2 (△ +0.4) 89.6 (△ +0.6) 82.8 (△ +0.5) 87.9 (△ +0.5) 
RoBERTa 91.4 ( +3.6) 83.1 ( +3.9) 89.3 ( +4.8) 82.7 ( +7.9) 87.9 ( +4.4) 
91.7 (△ +0.3) 83.2 (△ +0.1) 89.7 (△ +0.4) 83.4 (△ +0.7) 88.0 (△ +0.1) 
XLNet 91.5 ( +3.7) 84.1 ( +4.9) 89.8 ( +5.3) 85.2 ( +10.4) 88.2 ( +4.7) 
91.6 (△ +0.1) 84.2 (△ +0.1) 89.8 (△ +0.0) 85.4 (△ +0.2) 88.3 (△ +0.1) 
ALBERT 91.6 ( +3.8) 84.0 ( +4.8) 90.0 ( +5.5) 84.9 ( +10.1) 88.5 ( +5.0) 
91.6 (△ +0.0) 84.3 (△ +0.3) 90.1 (△ +0.1) 85.1 (△ +0.2) 88.7 (△ +0.2) 
SystemSYNCoNLL09 WSJCoNLL09 BrownCoNLL05 WSJCoNLL05 BrownCoNLL12
Baseline 87.8 79.2 84.5 74.8 83.5 
89.2 (△ +1.4) 80.1 (△ +0.9) 85.6 (△ +1.1) 76.2 (△ +1.4) 84.7 (△ +1.2) 
ELMo 90.4 ( +2.6) 81.5 ( +2.3) 87.7 ( +3.2) 80.5 ( +5.7) 86.0 ( +2.5) 
91.1 (△ +0.7) 82.1 (△ +0.6) 88.6 (△ +0.9) 81.0 (△ +0.5) 86.7 (△ +0.7) 
BERT 91.4 ( +3.6) 82.8 ( +3.6) 89.0 ( +4.5) 82.3 ( +7.5) 87.4 ( +3.9) 
91.8 (△ +0.4) 83.2 (△ +0.4) 89.6 (△ +0.6) 82.8 (△ +0.5) 87.9 (△ +0.5) 
RoBERTa 91.4 ( +3.6) 83.1 ( +3.9) 89.3 ( +4.8) 82.7 ( +7.9) 87.9 ( +4.4) 
91.7 (△ +0.3) 83.2 (△ +0.1) 89.7 (△ +0.4) 83.4 (△ +0.7) 88.0 (△ +0.1) 
XLNet 91.5 ( +3.7) 84.1 ( +4.9) 89.8 ( +5.3) 85.2 ( +10.4) 88.2 ( +4.7) 
91.6 (△ +0.1) 84.2 (△ +0.1) 89.8 (△ +0.0) 85.4 (△ +0.2) 88.3 (△ +0.1) 
ALBERT 91.6 ( +3.8) 84.0 ( +4.8) 90.0 ( +5.5) 84.9 ( +10.1) 88.5 ( +5.0) 
91.6 (△ +0.0) 84.3 (△ +0.3) 90.1 (△ +0.1) 85.1 (△ +0.2) 88.7 (△ +0.2) 
Close Modal

or Create an Account

Close Modal
Close Modal