Skip to Main Content
Table 4

Span SRL results with pre-identified predicates on the CoNLL-2005 and CoNLL-2012 test sets. The “PLM” column indicates whether and which pre-trained language model is used, the “SYN” column indicates whether syntax information is employed, and “+E” in the “PLM” column shows that the model leverages the ELMo for pre-trained language model features. [Ens.] is used to specify the ensemble system and [Joint] means joint learning with other tasks.

SystemPLMSYNCoNLL05 WSJCoNLL05 BrownCoNLL12
PRF1PRF1PRF1
[Ens.] (Punyakanok, Roth, and Yih 2008)  82.3 76.8 79.4 73.4 62.9 67.8 − − − 
(Toutanova, Haghighi, and Manning 2008)  − − 79.7 − − 67.8 − − − 
[Ens.] (Toutanova, Haghighi, and Manning 2008)  81.9 78.8 80.3 − − 68.8 − − − 
(Pradhan et al. 2013)*  − − − − − − 78.5 76.6 77.5 
(Täckström, Ganchev, and Das 2015)  82.3 77.6 79.9 74.3 68.6 71.3 80.6 78.2 79.4 
(Zhou and Xu 2015)  82.9 82.8 82.8 70.7 68.2 69.4 − − 81.3 
(FitzGerald et al. 2015)  81.8 77.3 79.4 73.8 68.8 71.2 80.9 78.4 79.6 
[Ens.] (FitzGerald et al. 2015)  82.5 78.2 80.3 74.5 70.0 72.2 81.2 79.0 80.1 
(He et al. 2017)  83.1 83.0 83.1 72.9 71.4 72.1 81.7 81.6 81.7 
[Ens.] (He et al. 2017)  85.0 84.3 84.6 74.9 72.4 73.6 83.5 83.3 83.4 
(Yang and Mitchell 2017)  − − 81.9 − − 72.0 − − − 
(Tan et al. 2018)  84.5 85.2 84.8 73.5 74.6 74.1 81.9 83.6 82.7 
[Ens.] (Tan et al. 2018)  85.9 86.3 86.1 74.6 75.0 74.8 83.3 84.5 83.9 
 
(Peters et al. 2018)  − − − − − − − − 81.4 
+E − − − − − − − − 84.6 
 
(He et al. 2018a)  − − 83.9 − − 73.7 − − 82.1 
+E − − 87.4 − − 80.4 − − 85.5 
 
(Strubell et al. 2018)  84.7 84.2 84.5 73.9 72.4 73.1 − − − 
84.6 84.6 84.6 74.8 74.3 74.6 − − − 
 
(Ouchi, Shindo, and Matsumoto 2018)  84.7 82.3 83.5 76.0 70.4 73.1 84.4 81.7 83.0 
+E 88.2 87.0 87.6 79.9 77.5 78.7 87.1 85.3 86.2 
 
(Wang et al. 2019) +E − − 87.7 − − 78.1 − − 85.8 
+E − − 88.2 − − 79.3 − − 86.4 
 
(Marcheggiani and Titov 2020)  85.8 85.1 85.4 76.2 74.7 75.5 84.5 84.3 84.4 
 
[Joint] (Zhou, Li, and Zhao 2020)  85.9 85.8 85.8 76.9 74.6 75.7 − − − 
+E 87.8 88.3 88.0 79.6 78.6 79.1 − − − 
 
Sequence-based +E 87.4 85.6 86.5 80.0 78.1 79.0 84.2 85.6 84.9 
+GCN Syntax Encoder +E 87.2 86.8 87.0 78.6 80.2 79.4 85.3 85.7 85.5 
+SA-LSTM Syntax Encoder +E 87.1 86.5 86.8 79.3 78.9 79.1 85.9 84.3 85.1 
+Tree-LSTM Syntax Encoder +E 87.1 85.9 86.5 78.8 79.2 79.0 85.2 84.2 84.7 
 
Tree-based +E 88.8 86.0 87.4 79.9 79.5 79.7 86.6 84.8 85.7 
+GCN Syntax Encoder +E 87.7 88.3 88.0 81.1 79.9 80.5 86.9 85.5 86.2 
+SA-LSTM Syntax Encoder +E 87.5 87.6 87.6 80.4 79.8 80.1 86.3 85.3 85.8 
+Tree-LSTM Syntax Encoder +E 87.0 87.6 87.3 81.0 79.1 80.0 86.0 85.6 85.8 
 
Graph-based (2019a) +E 87.9 87.5 87.7 80.6 80.4 80.5 85.7 86.3 86.0 
+Constituent Soft Pruning +E 88.4 87.4 87.9 80.9 80.3 80.6 85.5 86.9 86.2 
+GCN Syntax Encoder +E 89.0 88.2 88.6 80.8 81.2 81.0 87.2 86.2 86.7 
+SA-LSTM Syntax Encoder +E 88.6 87.8 88.2 81.0 81.2 81.1 87.0 85.8 86.4 
+Tree-LSTM Syntax Encoder +E 86.9 89.1 88.0 81.5 80.3 80.9 86.6 86.0 86.3 
SystemPLMSYNCoNLL05 WSJCoNLL05 BrownCoNLL12
PRF1PRF1PRF1
[Ens.] (Punyakanok, Roth, and Yih 2008)  82.3 76.8 79.4 73.4 62.9 67.8 − − − 
(Toutanova, Haghighi, and Manning 2008)  − − 79.7 − − 67.8 − − − 
[Ens.] (Toutanova, Haghighi, and Manning 2008)  81.9 78.8 80.3 − − 68.8 − − − 
(Pradhan et al. 2013)*  − − − − − − 78.5 76.6 77.5 
(Täckström, Ganchev, and Das 2015)  82.3 77.6 79.9 74.3 68.6 71.3 80.6 78.2 79.4 
(Zhou and Xu 2015)  82.9 82.8 82.8 70.7 68.2 69.4 − − 81.3 
(FitzGerald et al. 2015)  81.8 77.3 79.4 73.8 68.8 71.2 80.9 78.4 79.6 
[Ens.] (FitzGerald et al. 2015)  82.5 78.2 80.3 74.5 70.0 72.2 81.2 79.0 80.1 
(He et al. 2017)  83.1 83.0 83.1 72.9 71.4 72.1 81.7 81.6 81.7 
[Ens.] (He et al. 2017)  85.0 84.3 84.6 74.9 72.4 73.6 83.5 83.3 83.4 
(Yang and Mitchell 2017)  − − 81.9 − − 72.0 − − − 
(Tan et al. 2018)  84.5 85.2 84.8 73.5 74.6 74.1 81.9 83.6 82.7 
[Ens.] (Tan et al. 2018)  85.9 86.3 86.1 74.6 75.0 74.8 83.3 84.5 83.9 
 
(Peters et al. 2018)  − − − − − − − − 81.4 
+E − − − − − − − − 84.6 
 
(He et al. 2018a)  − − 83.9 − − 73.7 − − 82.1 
+E − − 87.4 − − 80.4 − − 85.5 
 
(Strubell et al. 2018)  84.7 84.2 84.5 73.9 72.4 73.1 − − − 
84.6 84.6 84.6 74.8 74.3 74.6 − − − 
 
(Ouchi, Shindo, and Matsumoto 2018)  84.7 82.3 83.5 76.0 70.4 73.1 84.4 81.7 83.0 
+E 88.2 87.0 87.6 79.9 77.5 78.7 87.1 85.3 86.2 
 
(Wang et al. 2019) +E − − 87.7 − − 78.1 − − 85.8 
+E − − 88.2 − − 79.3 − − 86.4 
 
(Marcheggiani and Titov 2020)  85.8 85.1 85.4 76.2 74.7 75.5 84.5 84.3 84.4 
 
[Joint] (Zhou, Li, and Zhao 2020)  85.9 85.8 85.8 76.9 74.6 75.7 − − − 
+E 87.8 88.3 88.0 79.6 78.6 79.1 − − − 
 
Sequence-based +E 87.4 85.6 86.5 80.0 78.1 79.0 84.2 85.6 84.9 
+GCN Syntax Encoder +E 87.2 86.8 87.0 78.6 80.2 79.4 85.3 85.7 85.5 
+SA-LSTM Syntax Encoder +E 87.1 86.5 86.8 79.3 78.9 79.1 85.9 84.3 85.1 
+Tree-LSTM Syntax Encoder +E 87.1 85.9 86.5 78.8 79.2 79.0 85.2 84.2 84.7 
 
Tree-based +E 88.8 86.0 87.4 79.9 79.5 79.7 86.6 84.8 85.7 
+GCN Syntax Encoder +E 87.7 88.3 88.0 81.1 79.9 80.5 86.9 85.5 86.2 
+SA-LSTM Syntax Encoder +E 87.5 87.6 87.6 80.4 79.8 80.1 86.3 85.3 85.8 
+Tree-LSTM Syntax Encoder +E 87.0 87.6 87.3 81.0 79.1 80.0 86.0 85.6 85.8 
 
Graph-based (2019a) +E 87.9 87.5 87.7 80.6 80.4 80.5 85.7 86.3 86.0 
+Constituent Soft Pruning +E 88.4 87.4 87.9 80.9 80.3 80.6 85.5 86.9 86.2 
+GCN Syntax Encoder +E 89.0 88.2 88.6 80.8 81.2 81.0 87.2 86.2 86.7 
+SA-LSTM Syntax Encoder +E 88.6 87.8 88.2 81.0 81.2 81.1 87.0 85.8 86.4 
+Tree-LSTM Syntax Encoder +E 86.9 89.1 88.0 81.5 80.3 80.9 86.6 86.0 86.3 
Close Modal

or Create an Account

Close Modal
Close Modal