Span SRL results with pre-identified predicates on the CoNLL-2005 and CoNLL-2012 test sets. The “PLM” column indicates whether and which pre-trained language model is used, the “SYN” column indicates whether syntax information is employed, and “+E” in the “PLM” column shows that the model leverages the ELMo for pre-trained language model features. [Ens.] is used to specify the ensemble system and [Joint] means joint learning with other tasks.
| System . | PLM . | SYN . | CoNLL05 WSJ . | CoNLL05 Brown . | CoNLL12 . | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| P . | R . | F1 . | P . | R . | F1 . | P . | R . | F1 . | |||
| [Ens.] (Punyakanok, Roth, and Yih 2008) | Y | 82.3 | 76.8 | 79.4 | 73.4 | 62.9 | 67.8 | − | − | − | |
| (Toutanova, Haghighi, and Manning 2008) | Y | − | − | 79.7 | − | − | 67.8 | − | − | − | |
| [Ens.] (Toutanova, Haghighi, and Manning 2008) | Y | 81.9 | 78.8 | 80.3 | − | − | 68.8 | − | − | − | |
| (Pradhan et al. 2013)* | Y | − | − | − | − | − | − | 78.5 | 76.6 | 77.5 | |
| (Täckström, Ganchev, and Das 2015) | Y | 82.3 | 77.6 | 79.9 | 74.3 | 68.6 | 71.3 | 80.6 | 78.2 | 79.4 | |
| (Zhou and Xu 2015) | N | 82.9 | 82.8 | 82.8 | 70.7 | 68.2 | 69.4 | − | − | 81.3 | |
| (FitzGerald et al. 2015) | Y | 81.8 | 77.3 | 79.4 | 73.8 | 68.8 | 71.2 | 80.9 | 78.4 | 79.6 | |
| [Ens.] (FitzGerald et al. 2015) | Y | 82.5 | 78.2 | 80.3 | 74.5 | 70.0 | 72.2 | 81.2 | 79.0 | 80.1 | |
| (He et al. 2017) | Y | 83.1 | 83.0 | 83.1 | 72.9 | 71.4 | 72.1 | 81.7 | 81.6 | 81.7 | |
| [Ens.] (He et al. 2017) | Y | 85.0 | 84.3 | 84.6 | 74.9 | 72.4 | 73.6 | 83.5 | 83.3 | 83.4 | |
| (Yang and Mitchell 2017) | N | − | − | 81.9 | − | − | 72.0 | − | − | − | |
| (Tan et al. 2018) | N | 84.5 | 85.2 | 84.8 | 73.5 | 74.6 | 74.1 | 81.9 | 83.6 | 82.7 | |
| [Ens.] (Tan et al. 2018) | N | 85.9 | 86.3 | 86.1 | 74.6 | 75.0 | 74.8 | 83.3 | 84.5 | 83.9 | |
| (Peters et al. 2018) | N | − | − | − | − | − | − | − | − | 81.4 | |
| +E | N | − | − | − | − | − | − | − | − | 84.6 | |
| (He et al. 2018a) | N | − | − | 83.9 | − | − | 73.7 | − | − | 82.1 | |
| +E | N | − | − | 87.4 | − | − | 80.4 | − | − | 85.5 | |
| (Strubell et al. 2018) | N | 84.7 | 84.2 | 84.5 | 73.9 | 72.4 | 73.1 | − | − | − | |
| Y | 84.6 | 84.6 | 84.6 | 74.8 | 74.3 | 74.6 | − | − | − | ||
| (Ouchi, Shindo, and Matsumoto 2018) | N | 84.7 | 82.3 | 83.5 | 76.0 | 70.4 | 73.1 | 84.4 | 81.7 | 83.0 | |
| +E | N | 88.2 | 87.0 | 87.6 | 79.9 | 77.5 | 78.7 | 87.1 | 85.3 | 86.2 | |
| (Wang et al. 2019) | +E | N | − | − | 87.7 | − | − | 78.1 | − | − | 85.8 |
| +E | Y | − | − | 88.2 | − | − | 79.3 | − | − | 86.4 | |
| (Marcheggiani and Titov 2020) | Y | 85.8 | 85.1 | 85.4 | 76.2 | 74.7 | 75.5 | 84.5 | 84.3 | 84.4 | |
| [Joint] (Zhou, Li, and Zhao 2020) | N | 85.9 | 85.8 | 85.8 | 76.9 | 74.6 | 75.7 | − | − | − | |
| +E | N | 87.8 | 88.3 | 88.0 | 79.6 | 78.6 | 79.1 | − | − | − | |
| Sequence-based | +E | N | 87.4 | 85.6 | 86.5 | 80.0 | 78.1 | 79.0 | 84.2 | 85.6 | 84.9 |
| +GCN Syntax Encoder | +E | Y | 87.2 | 86.8 | 87.0 | 78.6 | 80.2 | 79.4 | 85.3 | 85.7 | 85.5 |
| +SA-LSTM Syntax Encoder | +E | Y | 87.1 | 86.5 | 86.8 | 79.3 | 78.9 | 79.1 | 85.9 | 84.3 | 85.1 |
| +Tree-LSTM Syntax Encoder | +E | Y | 87.1 | 85.9 | 86.5 | 78.8 | 79.2 | 79.0 | 85.2 | 84.2 | 84.7 |
| Tree-based | +E | N | 88.8 | 86.0 | 87.4 | 79.9 | 79.5 | 79.7 | 86.6 | 84.8 | 85.7 |
| +GCN Syntax Encoder | +E | Y | 87.7 | 88.3 | 88.0 | 81.1 | 79.9 | 80.5 | 86.9 | 85.5 | 86.2 |
| +SA-LSTM Syntax Encoder | +E | Y | 87.5 | 87.6 | 87.6 | 80.4 | 79.8 | 80.1 | 86.3 | 85.3 | 85.8 |
| +Tree-LSTM Syntax Encoder | +E | Y | 87.0 | 87.6 | 87.3 | 81.0 | 79.1 | 80.0 | 86.0 | 85.6 | 85.8 |
| Graph-based (2019a) | +E | N | 87.9 | 87.5 | 87.7 | 80.6 | 80.4 | 80.5 | 85.7 | 86.3 | 86.0 |
| +Constituent Soft Pruning | +E | Y | 88.4 | 87.4 | 87.9 | 80.9 | 80.3 | 80.6 | 85.5 | 86.9 | 86.2 |
| +GCN Syntax Encoder | +E | Y | 89.0 | 88.2 | 88.6 | 80.8 | 81.2 | 81.0 | 87.2 | 86.2 | 86.7 |
| +SA-LSTM Syntax Encoder | +E | Y | 88.6 | 87.8 | 88.2 | 81.0 | 81.2 | 81.1 | 87.0 | 85.8 | 86.4 |
| +Tree-LSTM Syntax Encoder | +E | Y | 86.9 | 89.1 | 88.0 | 81.5 | 80.3 | 80.9 | 86.6 | 86.0 | 86.3 |
| System . | PLM . | SYN . | CoNLL05 WSJ . | CoNLL05 Brown . | CoNLL12 . | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| P . | R . | F1 . | P . | R . | F1 . | P . | R . | F1 . | |||
| [Ens.] (Punyakanok, Roth, and Yih 2008) | Y | 82.3 | 76.8 | 79.4 | 73.4 | 62.9 | 67.8 | − | − | − | |
| (Toutanova, Haghighi, and Manning 2008) | Y | − | − | 79.7 | − | − | 67.8 | − | − | − | |
| [Ens.] (Toutanova, Haghighi, and Manning 2008) | Y | 81.9 | 78.8 | 80.3 | − | − | 68.8 | − | − | − | |
| (Pradhan et al. 2013)* | Y | − | − | − | − | − | − | 78.5 | 76.6 | 77.5 | |
| (Täckström, Ganchev, and Das 2015) | Y | 82.3 | 77.6 | 79.9 | 74.3 | 68.6 | 71.3 | 80.6 | 78.2 | 79.4 | |
| (Zhou and Xu 2015) | N | 82.9 | 82.8 | 82.8 | 70.7 | 68.2 | 69.4 | − | − | 81.3 | |
| (FitzGerald et al. 2015) | Y | 81.8 | 77.3 | 79.4 | 73.8 | 68.8 | 71.2 | 80.9 | 78.4 | 79.6 | |
| [Ens.] (FitzGerald et al. 2015) | Y | 82.5 | 78.2 | 80.3 | 74.5 | 70.0 | 72.2 | 81.2 | 79.0 | 80.1 | |
| (He et al. 2017) | Y | 83.1 | 83.0 | 83.1 | 72.9 | 71.4 | 72.1 | 81.7 | 81.6 | 81.7 | |
| [Ens.] (He et al. 2017) | Y | 85.0 | 84.3 | 84.6 | 74.9 | 72.4 | 73.6 | 83.5 | 83.3 | 83.4 | |
| (Yang and Mitchell 2017) | N | − | − | 81.9 | − | − | 72.0 | − | − | − | |
| (Tan et al. 2018) | N | 84.5 | 85.2 | 84.8 | 73.5 | 74.6 | 74.1 | 81.9 | 83.6 | 82.7 | |
| [Ens.] (Tan et al. 2018) | N | 85.9 | 86.3 | 86.1 | 74.6 | 75.0 | 74.8 | 83.3 | 84.5 | 83.9 | |
| (Peters et al. 2018) | N | − | − | − | − | − | − | − | − | 81.4 | |
| +E | N | − | − | − | − | − | − | − | − | 84.6 | |
| (He et al. 2018a) | N | − | − | 83.9 | − | − | 73.7 | − | − | 82.1 | |
| +E | N | − | − | 87.4 | − | − | 80.4 | − | − | 85.5 | |
| (Strubell et al. 2018) | N | 84.7 | 84.2 | 84.5 | 73.9 | 72.4 | 73.1 | − | − | − | |
| Y | 84.6 | 84.6 | 84.6 | 74.8 | 74.3 | 74.6 | − | − | − | ||
| (Ouchi, Shindo, and Matsumoto 2018) | N | 84.7 | 82.3 | 83.5 | 76.0 | 70.4 | 73.1 | 84.4 | 81.7 | 83.0 | |
| +E | N | 88.2 | 87.0 | 87.6 | 79.9 | 77.5 | 78.7 | 87.1 | 85.3 | 86.2 | |
| (Wang et al. 2019) | +E | N | − | − | 87.7 | − | − | 78.1 | − | − | 85.8 |
| +E | Y | − | − | 88.2 | − | − | 79.3 | − | − | 86.4 | |
| (Marcheggiani and Titov 2020) | Y | 85.8 | 85.1 | 85.4 | 76.2 | 74.7 | 75.5 | 84.5 | 84.3 | 84.4 | |
| [Joint] (Zhou, Li, and Zhao 2020) | N | 85.9 | 85.8 | 85.8 | 76.9 | 74.6 | 75.7 | − | − | − | |
| +E | N | 87.8 | 88.3 | 88.0 | 79.6 | 78.6 | 79.1 | − | − | − | |
| Sequence-based | +E | N | 87.4 | 85.6 | 86.5 | 80.0 | 78.1 | 79.0 | 84.2 | 85.6 | 84.9 |
| +GCN Syntax Encoder | +E | Y | 87.2 | 86.8 | 87.0 | 78.6 | 80.2 | 79.4 | 85.3 | 85.7 | 85.5 |
| +SA-LSTM Syntax Encoder | +E | Y | 87.1 | 86.5 | 86.8 | 79.3 | 78.9 | 79.1 | 85.9 | 84.3 | 85.1 |
| +Tree-LSTM Syntax Encoder | +E | Y | 87.1 | 85.9 | 86.5 | 78.8 | 79.2 | 79.0 | 85.2 | 84.2 | 84.7 |
| Tree-based | +E | N | 88.8 | 86.0 | 87.4 | 79.9 | 79.5 | 79.7 | 86.6 | 84.8 | 85.7 |
| +GCN Syntax Encoder | +E | Y | 87.7 | 88.3 | 88.0 | 81.1 | 79.9 | 80.5 | 86.9 | 85.5 | 86.2 |
| +SA-LSTM Syntax Encoder | +E | Y | 87.5 | 87.6 | 87.6 | 80.4 | 79.8 | 80.1 | 86.3 | 85.3 | 85.8 |
| +Tree-LSTM Syntax Encoder | +E | Y | 87.0 | 87.6 | 87.3 | 81.0 | 79.1 | 80.0 | 86.0 | 85.6 | 85.8 |
| Graph-based (2019a) | +E | N | 87.9 | 87.5 | 87.7 | 80.6 | 80.4 | 80.5 | 85.7 | 86.3 | 86.0 |
| +Constituent Soft Pruning | +E | Y | 88.4 | 87.4 | 87.9 | 80.9 | 80.3 | 80.6 | 85.5 | 86.9 | 86.2 |
| +GCN Syntax Encoder | +E | Y | 89.0 | 88.2 | 88.6 | 80.8 | 81.2 | 81.0 | 87.2 | 86.2 | 86.7 |
| +SA-LSTM Syntax Encoder | +E | Y | 88.6 | 87.8 | 88.2 | 81.0 | 81.2 | 81.1 | 87.0 | 85.8 | 86.4 |
| +Tree-LSTM Syntax Encoder | +E | Y | 86.9 | 89.1 | 88.0 | 81.5 | 80.3 | 80.9 | 86.6 | 86.0 | 86.3 |