Skip to Main Content
Table 6

Performance (F1) of GloVe word-based BiLSTM-CRF and BERT. System 1 denotes the oracle combination of separate systems which access to specific input representation only. System 2 refers to a single system with access to all the input of the various systems in system 1. The first two panels were trained on the Original English CoNLL 03 training data and tested on the original English CoNLL 03 test data and the WikiGold data. The last panel was trained and tested on the respective splits of MUC-6.

System 1 (Oracle)System 2CoNLLWikipediaMUC-6
sys 1sys 2sys 1sys 2sys 1sys 2
FW context – BW context LSTM-CRF Bi context LSTM-CRF 75.3 59.8 49.3 30.2 85.3 61.1 
  
FW context – BW context – GloVe fine-tuned LSTM-CRF Full LSTM-CRF 92.2 91.0 72.4 63.6 94.9 90.9 
  
Full system – FW context – BW context – GloVe fine-tuned LSTM-CRF Full LSTM-CRF 95.1 91.0 76.8 63.6 96.1 90.9 
  
word-only – context-only BERT Full BERT 92.5 92.5 86.8 75.2 93.7 96.7 
  
Full – word-only – context-only BERT Full BERT 96.5 92.5 90.1 75.2 98.5 96.7 
System 1 (Oracle)System 2CoNLLWikipediaMUC-6
sys 1sys 2sys 1sys 2sys 1sys 2
FW context – BW context LSTM-CRF Bi context LSTM-CRF 75.3 59.8 49.3 30.2 85.3 61.1 
  
FW context – BW context – GloVe fine-tuned LSTM-CRF Full LSTM-CRF 92.2 91.0 72.4 63.6 94.9 90.9 
  
Full system – FW context – BW context – GloVe fine-tuned LSTM-CRF Full LSTM-CRF 95.1 91.0 76.8 63.6 96.1 90.9 
  
word-only – context-only BERT Full BERT 92.5 92.5 86.8 75.2 93.7 96.7 
  
Full – word-only – context-only BERT Full BERT 96.5 92.5 90.1 75.2 98.5 96.7 
Close Modal

or Create an Account

Close Modal
Close Modal