Skip to Main Content
Table 2

Dependency SRL Results with pre-identified predicates (w/ pred) and without pre-identified predicates (w/o pred) on the CoNLL-2009 English in-domain (WSJ) and out-of-domain (Brown) test sets. The “PLM” column indicates whether and which pre-trained language model is used, the “SYN” column indicates whether syntax information is employed, and “+E” in the “PLM” column shows that the model leverages ELMo for pre-trained language model features. [Ens.] is used to specify the ensemble system, [Semi.] indicates semi-supervised training is adopted, and [Joint] means joint learning with other tasks.

SystemPLMSYNw/ predw/o pred
WSJBrownWSJBrown
PRF1PRF1PRF1PRF1
(Zhao et al. 2009b)  − − 85.4 − − 73.3 − − − − − − 
(Zhao et al. 2009a)  − − 86.2 − − 74.6 − − − − − − 
(Lei et al. 2015)  − − 86.6 − − 75.6 − − − − − − 
(FitzGerald et al. 2015)  − − 87.3 − − 75.2 − − − − − − 
[Ens.] (FitzGerald et al. 2015)  − − 87.8 − − 75.5 − − − − − − 
(Roth and Lapata 2016)  90.0 85.5 87.7 78.6 73.8 76.1 − − − − − − 
[Ens.] (Roth and Lapata 2016)  90.3 85.7 87.9 79.7 73.6 76.5 − − − − − − 
(Swayamdipta et al. 2016)  − − 85.0 − − − − − 80.5 − − − 
(Marcheggiani and Titov 2017)  89.1 86.8 88.0 78.5 75.9 77.2 − − − − − − 
[Ens.] (Marcheggiani and Titov 2017)  90.5 87.7 89.1 80.8 77.1 78.9 − − − − − − 
(Marcheggiani, Frolov, and Titov 2017)  88.7 86.8 87.7 79.4 76.2 77.7 − − − − − − 
(Mulcaire, Swayamdipta, and Smith 2018)  − − 87.2 − − − − − − − − − 
 
(Kasai et al. 2019)  89.0 88.2 88.6 78.0 77.2 77.6 − − − − − − 
+E 90.3 90.0 90.2 81.0 80.5 80.8 − − − − − − 
 
(Cai and Lapata 2019a)  91.1 90.4 90.7 82.1 81.3 81.6 − − − − − − 
[Semi.] (Cai and Lapata 2019a)  91.7 90.8 91.2 83.2 81.9 82.5 − − − − − − 
(Zhang, Wang, and Si 2019)  89.6 86.0 87.7 − − − − − − − − − 
(Lyu, Cohen, and Titov 2019) +E − − 91.0 − − 82.2 − − − − − − 
(Chen, Lyu, and Titov 2019) +E 90.7 91.4 91.1 82.7 82.8 82.7 − − − − − − 
 
[Joint] (Zhou, Li, and Zhao 2020)  88.7 89.8 89.3 82.5 83.2 82.8 84.2 87.6 85.9 76.5 78.5 77.5 
+E 89.7 90.9 90.3 83.9 85.0 84.5 85.2 88.2 86.7 78.6 80.8 79.7 
 
Sequence-based (2018b; 2018) +E 89.5 87.9 88.7 81.7 76.1 78.8 83.5 82.4 82.9 71.5 70.9 71.2 
+K-order Hard Pruning (2018b) +E 89.7 89.3 89.5 81.9 76.9 79.3 83.9 82.7 83.3 71.5 71.3 71.4 
+SynRule Soft Pruning +E 89.9 89.1 89.5 78.8 81.2 80.0 82.9 84.3 83.6 70.9 72.1 71.5 
+GCN Syntax Encoder (2018) +E 90.3 89.3 89.8 80.6 79.0 79.8 85.3 82.5 83.9 71.9 71.5 71.7 
+SA-LSTM Syntax Encoder (2018) +E 90.8 88.6 89.7 81.0 78.2 79.6 85.3 82.6 84.0 71.8 71.6 71.7 
+Tree-LSTM Syntax Encoder (2018) +E 90.0 88.8 89.4 80.4 78.7 79.5 83.1 83.7 83.4 70.9 72.1 71.5 
 
Tree-based (2018) +E 89.2 90.4 89.8 80.0 78.6 79.3 84.8 85.4 85.1 72.4 74.0 73.2 
+K-order Hard Pruning +E 90.3 89.5 89.9 80.0 79.0 79.5 83.9 86.5 85.2 73.6 72.8 73.2 
+SynRule Soft Pruning (2019) +E 90.0 90.7 90.3 79.6 80.4 80.0 84.9 85.9 85.4 72.7 74.3 73.5 
+GCN Syntax Encoder +E 90.9 90.1 90.5 81.4 78.8 80.1 86.1 84.9 85.5 73.5 73.7 73.6 
+SA-LSTM Syntax Encoder +E 91.1 89.9 90.5 80.9 79.5 80.2 85.3 85.0 85.2 72.9 73.5 73.2 
+Tree-LSTM Syntax Encoder +E 89.8 90.6 90.2 80.0 79.8 79.9 85.3 85.3 85.3 73.9 73.1 73.5 
 
Graph-based (2019a) +E 89.6 91.2 90.4 81.7 81.4 81.5 85.6 85.0 85.3 73.0 74.0 73.5 
+K-order Hard Pruning +E 90.3 89.7 90.0 80.7 81.9 81.3 84.6 85.8 85.2 73.7 73.3 73.5 
+SynRule Soft Pruning +E 89.8 90.6 90.2 80.8 82.4 81.6 85.0 86.0 85.5 72.8 74.4 73.6 
+GCN Syntax Encoder +E 90.5 91.7 91.1 83.3 80.9 82.1 86.2 86.0 86.1 73.8 74.6 74.2 
+SA-LSTM Syntax Encoder +E 91.0 90.4 90.7 82.4 81.6 82.0 86.3 85.5 85.9 75.4 72.8 74.1 
+Tree-LSTM Syntax Encoder +E 90.7 90.3 90.5 80.2 83.4 81.8 86.9 84.3 85.6 74.1 73.7 73.9 
SystemPLMSYNw/ predw/o pred
WSJBrownWSJBrown
PRF1PRF1PRF1PRF1
(Zhao et al. 2009b)  − − 85.4 − − 73.3 − − − − − − 
(Zhao et al. 2009a)  − − 86.2 − − 74.6 − − − − − − 
(Lei et al. 2015)  − − 86.6 − − 75.6 − − − − − − 
(FitzGerald et al. 2015)  − − 87.3 − − 75.2 − − − − − − 
[Ens.] (FitzGerald et al. 2015)  − − 87.8 − − 75.5 − − − − − − 
(Roth and Lapata 2016)  90.0 85.5 87.7 78.6 73.8 76.1 − − − − − − 
[Ens.] (Roth and Lapata 2016)  90.3 85.7 87.9 79.7 73.6 76.5 − − − − − − 
(Swayamdipta et al. 2016)  − − 85.0 − − − − − 80.5 − − − 
(Marcheggiani and Titov 2017)  89.1 86.8 88.0 78.5 75.9 77.2 − − − − − − 
[Ens.] (Marcheggiani and Titov 2017)  90.5 87.7 89.1 80.8 77.1 78.9 − − − − − − 
(Marcheggiani, Frolov, and Titov 2017)  88.7 86.8 87.7 79.4 76.2 77.7 − − − − − − 
(Mulcaire, Swayamdipta, and Smith 2018)  − − 87.2 − − − − − − − − − 
 
(Kasai et al. 2019)  89.0 88.2 88.6 78.0 77.2 77.6 − − − − − − 
+E 90.3 90.0 90.2 81.0 80.5 80.8 − − − − − − 
 
(Cai and Lapata 2019a)  91.1 90.4 90.7 82.1 81.3 81.6 − − − − − − 
[Semi.] (Cai and Lapata 2019a)  91.7 90.8 91.2 83.2 81.9 82.5 − − − − − − 
(Zhang, Wang, and Si 2019)  89.6 86.0 87.7 − − − − − − − − − 
(Lyu, Cohen, and Titov 2019) +E − − 91.0 − − 82.2 − − − − − − 
(Chen, Lyu, and Titov 2019) +E 90.7 91.4 91.1 82.7 82.8 82.7 − − − − − − 
 
[Joint] (Zhou, Li, and Zhao 2020)  88.7 89.8 89.3 82.5 83.2 82.8 84.2 87.6 85.9 76.5 78.5 77.5 
+E 89.7 90.9 90.3 83.9 85.0 84.5 85.2 88.2 86.7 78.6 80.8 79.7 
 
Sequence-based (2018b; 2018) +E 89.5 87.9 88.7 81.7 76.1 78.8 83.5 82.4 82.9 71.5 70.9 71.2 
+K-order Hard Pruning (2018b) +E 89.7 89.3 89.5 81.9 76.9 79.3 83.9 82.7 83.3 71.5 71.3 71.4 
+SynRule Soft Pruning +E 89.9 89.1 89.5 78.8 81.2 80.0 82.9 84.3 83.6 70.9 72.1 71.5 
+GCN Syntax Encoder (2018) +E 90.3 89.3 89.8 80.6 79.0 79.8 85.3 82.5 83.9 71.9 71.5 71.7 
+SA-LSTM Syntax Encoder (2018) +E 90.8 88.6 89.7 81.0 78.2 79.6 85.3 82.6 84.0 71.8 71.6 71.7 
+Tree-LSTM Syntax Encoder (2018) +E 90.0 88.8 89.4 80.4 78.7 79.5 83.1 83.7 83.4 70.9 72.1 71.5 
 
Tree-based (2018) +E 89.2 90.4 89.8 80.0 78.6 79.3 84.8 85.4 85.1 72.4 74.0 73.2 
+K-order Hard Pruning +E 90.3 89.5 89.9 80.0 79.0 79.5 83.9 86.5 85.2 73.6 72.8 73.2 
+SynRule Soft Pruning (2019) +E 90.0 90.7 90.3 79.6 80.4 80.0 84.9 85.9 85.4 72.7 74.3 73.5 
+GCN Syntax Encoder +E 90.9 90.1 90.5 81.4 78.8 80.1 86.1 84.9 85.5 73.5 73.7 73.6 
+SA-LSTM Syntax Encoder +E 91.1 89.9 90.5 80.9 79.5 80.2 85.3 85.0 85.2 72.9 73.5 73.2 
+Tree-LSTM Syntax Encoder +E 89.8 90.6 90.2 80.0 79.8 79.9 85.3 85.3 85.3 73.9 73.1 73.5 
 
Graph-based (2019a) +E 89.6 91.2 90.4 81.7 81.4 81.5 85.6 85.0 85.3 73.0 74.0 73.5 
+K-order Hard Pruning +E 90.3 89.7 90.0 80.7 81.9 81.3 84.6 85.8 85.2 73.7 73.3 73.5 
+SynRule Soft Pruning +E 89.8 90.6 90.2 80.8 82.4 81.6 85.0 86.0 85.5 72.8 74.4 73.6 
+GCN Syntax Encoder +E 90.5 91.7 91.1 83.3 80.9 82.1 86.2 86.0 86.1 73.8 74.6 74.2 
+SA-LSTM Syntax Encoder +E 91.0 90.4 90.7 82.4 81.6 82.0 86.3 85.5 85.9 75.4 72.8 74.1 
+Tree-LSTM Syntax Encoder +E 90.7 90.3 90.5 80.2 83.4 81.8 86.9 84.3 85.6 74.1 73.7 73.9 
Close Modal

or Create an Account

Close Modal
Close Modal