Skip to Main Content
Table 8.

Evaluation scores obtained for existing citation classification data sets

Data set# InstancesClassifierTask# classesMetricScore
Teufel et al. (2006b)  2,829 kNN (k = 3) Purpose 12 Macro-F 0.57 
Kappa 0.57 
Macro-F 0.68 
Kappa 0.59 
Polarity Macro-F 0.71 
Kappa 0.58 
Dong and Schäfer (2011)  1,768 NB Purpose Macro-F 0.66 
SVM 0.79 
Li et al. (2013)  6,355 MaxEnt Purpose 11 F-Score 0.67 
Abu-Jbara et al. (2013)  3,271 SVM Purpose Macro-F 0.58 
Accuracy 0.70 
Polarity Macro-F 0.71 
Accuracy 0.81 
CNN + Multidisciplinary embedding Purpose F-Score 0.79 
Polarity F-Score 0.82 
Hernández-Álvarez et al. (2017)  2,120 SVM Purpose F-score 0.89 
ROC Area 0.95 
Polarity F-score 0.93 
ROC Area 0.93 
348 Importance F-score 0.94 
Jurgens et al. (2018)  3,083 RF Purpose Macro-F 0.53 
Cohan et al. (2019)  11,020 biLSTM Attention + ELMO & structural scaffolds Purpose Macro-F 0.84 
SciBERT Purpose Macro-F 0.85 
Zhu et al. (2015)  3,143 NB Importance Macro-F 0.42 
Valenzuela et al. (2015)  450 SVM Importance Precision 0.65 
Recall 0.90 
Data set# InstancesClassifierTask# classesMetricScore
Teufel et al. (2006b)  2,829 kNN (k = 3) Purpose 12 Macro-F 0.57 
Kappa 0.57 
Macro-F 0.68 
Kappa 0.59 
Polarity Macro-F 0.71 
Kappa 0.58 
Dong and Schäfer (2011)  1,768 NB Purpose Macro-F 0.66 
SVM 0.79 
Li et al. (2013)  6,355 MaxEnt Purpose 11 F-Score 0.67 
Abu-Jbara et al. (2013)  3,271 SVM Purpose Macro-F 0.58 
Accuracy 0.70 
Polarity Macro-F 0.71 
Accuracy 0.81 
CNN + Multidisciplinary embedding Purpose F-Score 0.79 
Polarity F-Score 0.82 
Hernández-Álvarez et al. (2017)  2,120 SVM Purpose F-score 0.89 
ROC Area 0.95 
Polarity F-score 0.93 
ROC Area 0.93 
348 Importance F-score 0.94 
Jurgens et al. (2018)  3,083 RF Purpose Macro-F 0.53 
Cohan et al. (2019)  11,020 biLSTM Attention + ELMO & structural scaffolds Purpose Macro-F 0.84 
SciBERT Purpose Macro-F 0.85 
Zhu et al. (2015)  3,143 NB Importance Macro-F 0.42 
Valenzuela et al. (2015)  450 SVM Importance Precision 0.65 
Recall 0.90 
Close Modal

or Create an Account

Close Modal
Close Modal