Skip to Main Content
Table 6: 

Additional metrics of the neural models on the TNE test set. We report five metrics: the precision, recall, and F1 of the relation predictions, as well as the preposition accuracy on relations where the model predicted there is a relation (IPrep-Acc), as well as the accuracy where the model predicted there is no relation (UPrep-Acc). The first row is an estimated human agreement on 10% of the data, thus marked with an asterisk. These results are comparable with the ‘Pretrained’ part in Table 4.

Links-PLinks-RLinks-F1IPrep-AccUPrep-Acc
Human* 94.8 94.0 94.4 100.0 100.0 
Decoupled-static 67.3 19.6 30.4 77.7 54.2 
Decoupled-frozen-base 65.9 24.5 35.7 65.0 52.5 
Decoupled-frozen-large 67.2 22.8 34.1 68.2 52.7 
Decoupled-base 71.1 46.7 56.4 78.2 59.9 
Decoupled-large 73.5 47.2 57.5 79.3 61.5 
Coupled-static 70.1 17.0 27.4 85.0 49.8 
Coupled-frozen-base 73.8 10.6 18.5 81.5 44.0 
Coupled-frozen-large 73.3 14.4 24.0 79.8 43.7 
Coupled-base 76.4 52.4 62.2 79.1 50.7 
Coupled-large 80.5 53.1 64.0 81.8 49.6 
Links-PLinks-RLinks-F1IPrep-AccUPrep-Acc
Human* 94.8 94.0 94.4 100.0 100.0 
Decoupled-static 67.3 19.6 30.4 77.7 54.2 
Decoupled-frozen-base 65.9 24.5 35.7 65.0 52.5 
Decoupled-frozen-large 67.2 22.8 34.1 68.2 52.7 
Decoupled-base 71.1 46.7 56.4 78.2 59.9 
Decoupled-large 73.5 47.2 57.5 79.3 61.5 
Coupled-static 70.1 17.0 27.4 85.0 49.8 
Coupled-frozen-base 73.8 10.6 18.5 81.5 44.0 
Coupled-frozen-large 73.3 14.4 24.0 79.8 43.7 
Coupled-base 76.4 52.4 62.2 79.1 50.7 
Coupled-large 80.5 53.1 64.0 81.8 49.6 
Close Modal

or Create an Account

Close Modal
Close Modal