Performance (F1) of GloVe word-based BiLSTM-CRF and BERT. System 1 denotes the oracle combination of separate systems which access to specific input representation only. System 2 refers to a single system with access to all the input of the various systems in system 1. The first two panels were trained on the Original English CoNLL 03 training data and tested on the original English CoNLL 03 test data and the WikiGold data. The last panel was trained and tested on the respective splits of MUC-6.
| System 1 (Oracle) . | System 2 . | CoNLL . | Wikipedia . | MUC-6 . | |||
|---|---|---|---|---|---|---|---|
| sys 1 . | sys 2 . | sys 1 . | sys 2 . | sys 1 . | sys 2 . | ||
| FW context – BW context LSTM-CRF | Bi context LSTM-CRF | 75.3 | 59.8 | 49.3 | 30.2 | 85.3 | 61.1 |
| FW context – BW context – GloVe fine-tuned LSTM-CRF | Full LSTM-CRF | 92.2 | 91.0 | 72.4 | 63.6 | 94.9 | 90.9 |
| Full system – FW context – BW context – GloVe fine-tuned LSTM-CRF | Full LSTM-CRF | 95.1 | 91.0 | 76.8 | 63.6 | 96.1 | 90.9 |
| word-only – context-only BERT | Full BERT | 92.5 | 92.5 | 86.8 | 75.2 | 93.7 | 96.7 |
| Full – word-only – context-only BERT | Full BERT | 96.5 | 92.5 | 90.1 | 75.2 | 98.5 | 96.7 |
| System 1 (Oracle) . | System 2 . | CoNLL . | Wikipedia . | MUC-6 . | |||
|---|---|---|---|---|---|---|---|
| sys 1 . | sys 2 . | sys 1 . | sys 2 . | sys 1 . | sys 2 . | ||
| FW context – BW context LSTM-CRF | Bi context LSTM-CRF | 75.3 | 59.8 | 49.3 | 30.2 | 85.3 | 61.1 |
| FW context – BW context – GloVe fine-tuned LSTM-CRF | Full LSTM-CRF | 92.2 | 91.0 | 72.4 | 63.6 | 94.9 | 90.9 |
| Full system – FW context – BW context – GloVe fine-tuned LSTM-CRF | Full LSTM-CRF | 95.1 | 91.0 | 76.8 | 63.6 | 96.1 | 90.9 |
| word-only – context-only BERT | Full BERT | 92.5 | 92.5 | 86.8 | 75.2 | 93.7 | 96.7 |
| Full – word-only – context-only BERT | Full BERT | 96.5 | 92.5 | 90.1 | 75.2 | 98.5 | 96.7 |