Skip to Main Content
Table 3

Dependency SRL results on the CoNLL-2009 multilingual in-domain test sets with pre-identified predicates (w/ pred) setting. The first row is the best result of the CoNLL-2009 shared task (Hajič et al. 2009). The “PLM” column indicates whether and which pre-trained language model is used, the “SYN” column indicates whether syntax information is employed, “+E” in the “PLM” column indicates the model leverages pre-trained ELMo features for all languages, and “+E” in the “PLM” column indicates ELMo is only used for English.

SystemPLMSYNCACSDEENESJAZHAvg.
CoNLL-2009 best  80.3 86.5 79.7 86.2 80.5 78.3 78.6 81.4 
(Zhao et al. 2009a)  80.3 85.2 76.0 85.4 80.5 78.2 77.7 80.5 
(Roth and Lapata 2016)  − − 80.1 87.7 80.2 − 79.4 − 
(Marcheggiani and Titov 2017)  − − − 88.0 − − 82.5 − 
(Marcheggiani, Frolov, and Titov 2017)  − 86.0 − 87.7 80.3 − 81.2 − 
(Mulcaire, Swayamdipta, and Smith 2018)  79.5 85.1 70.0 87.2 77.3 76.0 81.9 79.6 
(Kasai et al. 2019) +E − − − 90.2 83.0 − −  
(Cai and Lapata 2019a)  − − 83.3 90.7 82.1 − 84.6 − 
[Semi.] (Cai and Lapata 2019a)  − − 83.8 91.2 82.9 − 85.0 − 
(Zhang, Wang, and Si 2019)  − − − 87.7 − − 84.2 − 
(Lyu, Cohen, and Titov 2019) +E 80.9 87.6 75.9 91.0 80.5 82.5 83.3 83.1 
(Chen, Lyu, and Titov 2019) +E 81.7 88.1 76.4 91.1 81.3 81.3 81.7 83.1 
 
Sequence-based (2018b; 2018) +E 84.0 87.8 76.8 88.7 82.9 82.8 83.1 83.7 
+K-order Hard Pruning (2018b) +E 84.5 88.3 77.3 89.5 83.3 82.9 82.8 84.1 
+SynRule Soft Pruning +E 84.4 88.2 77.5 89.5 83.2 83.0 83.3 84.2 
+GCN Syntax Encoder (2018) +E 84.6 88.5 77.2 89.8 83.6 83.2 83.8 84.4 
+SA-LSTM Syntax Encoder (2018) +E 84.3 88.5 77.0 89.7 83.5 83.1 83.5 84.2 
+Tree-LSTM Syntax Encoder (2018) +E 84.1 88.3 76.9 89.4 83.2 82.9 83.4 84.0 
 
Tree-based (2018) +E 84.1 88.4 78.4 89.9 83.5 83.0 84.0 84.5 
+K-order Hard Pruning +E 84.2 88.5 78.4 89.9 83.4 82.8 84.2 84.5 
+SynRule Soft Pruning (2019) +E 84.4 88.8 78.5 90.0 83.7 83.1 84.6 84.7 
+GCN Syntax Encoder +E 84.8 89.3 78.1 90.2 84.0 83.3 85.0 85.0 
+SA-LSTM Syntax Encoder +E 84.6 89.0 78.8 90.0 83.7 83.1 84.8 84.9 
+Tree-LSTM Syntax Encoder +E 84.4 88.9 78.6 89.9 83.6 83.0 84.5 84.7 
 
Graph-based (2019a) +E 85.0 90.2 76.0 90.0 83.8 82.7 85.7 84.8 
+K-order Hard Pruning +E 84.9 90.2 75.7 89.8 83.5 82.8 85.8 84.7 
+SynRule Soft Pruning +E 85.2 90.3 76.2 90.1 84.0 82.9 85.8 84.9 
+GCN Syntax Encoder +E 85.5 90.5 76.6 90.4 84.3 83.2 86.1 85.2 
+SA-LSTM Syntax Encoder +E 85.2 90.5 76.4 90.3 84.1 83.2 86.0 85.1 
+Tree-LSTM Syntax Encoder +E 85.0 90.3 76.2 90.3 84.0 83.0 85.8 84.9 
SystemPLMSYNCACSDEENESJAZHAvg.
CoNLL-2009 best  80.3 86.5 79.7 86.2 80.5 78.3 78.6 81.4 
(Zhao et al. 2009a)  80.3 85.2 76.0 85.4 80.5 78.2 77.7 80.5 
(Roth and Lapata 2016)  − − 80.1 87.7 80.2 − 79.4 − 
(Marcheggiani and Titov 2017)  − − − 88.0 − − 82.5 − 
(Marcheggiani, Frolov, and Titov 2017)  − 86.0 − 87.7 80.3 − 81.2 − 
(Mulcaire, Swayamdipta, and Smith 2018)  79.5 85.1 70.0 87.2 77.3 76.0 81.9 79.6 
(Kasai et al. 2019) +E − − − 90.2 83.0 − −  
(Cai and Lapata 2019a)  − − 83.3 90.7 82.1 − 84.6 − 
[Semi.] (Cai and Lapata 2019a)  − − 83.8 91.2 82.9 − 85.0 − 
(Zhang, Wang, and Si 2019)  − − − 87.7 − − 84.2 − 
(Lyu, Cohen, and Titov 2019) +E 80.9 87.6 75.9 91.0 80.5 82.5 83.3 83.1 
(Chen, Lyu, and Titov 2019) +E 81.7 88.1 76.4 91.1 81.3 81.3 81.7 83.1 
 
Sequence-based (2018b; 2018) +E 84.0 87.8 76.8 88.7 82.9 82.8 83.1 83.7 
+K-order Hard Pruning (2018b) +E 84.5 88.3 77.3 89.5 83.3 82.9 82.8 84.1 
+SynRule Soft Pruning +E 84.4 88.2 77.5 89.5 83.2 83.0 83.3 84.2 
+GCN Syntax Encoder (2018) +E 84.6 88.5 77.2 89.8 83.6 83.2 83.8 84.4 
+SA-LSTM Syntax Encoder (2018) +E 84.3 88.5 77.0 89.7 83.5 83.1 83.5 84.2 
+Tree-LSTM Syntax Encoder (2018) +E 84.1 88.3 76.9 89.4 83.2 82.9 83.4 84.0 
 
Tree-based (2018) +E 84.1 88.4 78.4 89.9 83.5 83.0 84.0 84.5 
+K-order Hard Pruning +E 84.2 88.5 78.4 89.9 83.4 82.8 84.2 84.5 
+SynRule Soft Pruning (2019) +E 84.4 88.8 78.5 90.0 83.7 83.1 84.6 84.7 
+GCN Syntax Encoder +E 84.8 89.3 78.1 90.2 84.0 83.3 85.0 85.0 
+SA-LSTM Syntax Encoder +E 84.6 89.0 78.8 90.0 83.7 83.1 84.8 84.9 
+Tree-LSTM Syntax Encoder +E 84.4 88.9 78.6 89.9 83.6 83.0 84.5 84.7 
 
Graph-based (2019a) +E 85.0 90.2 76.0 90.0 83.8 82.7 85.7 84.8 
+K-order Hard Pruning +E 84.9 90.2 75.7 89.8 83.5 82.8 85.8 84.7 
+SynRule Soft Pruning +E 85.2 90.3 76.2 90.1 84.0 82.9 85.8 84.9 
+GCN Syntax Encoder +E 85.5 90.5 76.6 90.4 84.3 83.2 86.1 85.2 
+SA-LSTM Syntax Encoder +E 85.2 90.5 76.4 90.3 84.1 83.2 86.0 85.1 
+Tree-LSTM Syntax Encoder +E 85.0 90.3 76.2 90.3 84.0 83.0 85.8 84.9 
Close Modal

or Create an Account

Close Modal
Close Modal