Identification systems: Measures used and results.
| System . | Precision . | Recall . | F-score . | Acc . | Lim. . | Open . |
|---|---|---|---|---|---|---|
| Mason (2004) | – | – | – | 0.77 | – | ✓ |
| Birke and Sarkar (2006) | – | – | 0.54 | – | – | ✓ |
| Gedigian et al. (2006) | – | – | – | 0.95* | ✓ | – |
| Krishnakumaran, Zhu (2007) | – | – | – | 0.58 | ✓ | – |
| Shutova et al. (2010) | 0.79 | – | – | – | – | ✓ |
| Li and Sporleder (2010) | – | – | 0.75 | 0.78 | – | ✓ |
| Turney et al. (2011) | – | – | 0.68** | 0.79** | – | ✓ |
| Neuman et al. (2013) | 0.71 | 0.43–0.97 | – | – | ✓ | – |
| Dunn (2013) | – | – | 0.58 | – | – | ✓ |
| Tsvetkov et al. (2013; 2014) | 0.78 | 0.79 | 0.78 | – | – | ✓ |
| Mohler et al. (2013) | 0.56 | 0.93 | 0.7 | – | ✓ | – |
| Hentz et al. (2013) | 0.54 | 0.64 | 0.59 | – | ✓ | – |
| Hovy et al. (2013) | 0.7 | 0.8 | 0.75 | 0.75 | – | ✓ |
| Wilks et al. (2013) | 0.57 | 0.82 | 0.67 | – | ✓ | – |
| Strzalkowski et al. (2013) | – | – | – | 0.71 | ✓ | – |
| Shutova and Sun (2013) | 0.65 (LM); | 0.61 (CM) | – | – | – | ✓ |
| 0.69 (CM) | ||||||
| Gandy et al. (2013) | 0.76 (LM); | 0.82 (LM) | – | – | ✓ | – |
| 0.65 (CM) | ||||||
| Li et al. (2013) | 0.65–0.73 | 0.52–0.66 | 0.58–0.69 | – | – | ✓ |
| Shutova (2013) | 0.68 | 0.66 | 0.67 | – | – | ✓ |
| System . | Precision . | Recall . | F-score . | Acc . | Lim. . | Open . |
|---|---|---|---|---|---|---|
| Mason (2004) | – | – | – | 0.77 | – | ✓ |
| Birke and Sarkar (2006) | – | – | 0.54 | – | – | ✓ |
| Gedigian et al. (2006) | – | – | – | 0.95* | ✓ | – |
| Krishnakumaran, Zhu (2007) | – | – | – | 0.58 | ✓ | – |
| Shutova et al. (2010) | 0.79 | – | – | – | – | ✓ |
| Li and Sporleder (2010) | – | – | 0.75 | 0.78 | – | ✓ |
| Turney et al. (2011) | – | – | 0.68** | 0.79** | – | ✓ |
| Neuman et al. (2013) | 0.71 | 0.43–0.97 | – | – | ✓ | – |
| Dunn (2013) | – | – | 0.58 | – | – | ✓ |
| Tsvetkov et al. (2013; 2014) | 0.78 | 0.79 | 0.78 | – | – | ✓ |
| Mohler et al. (2013) | 0.56 | 0.93 | 0.7 | – | ✓ | – |
| Hentz et al. (2013) | 0.54 | 0.64 | 0.59 | – | ✓ | – |
| Hovy et al. (2013) | 0.7 | 0.8 | 0.75 | 0.75 | – | ✓ |
| Wilks et al. (2013) | 0.57 | 0.82 | 0.67 | – | ✓ | – |
| Strzalkowski et al. (2013) | – | – | – | 0.71 | ✓ | – |
| Shutova and Sun (2013) | 0.65 (LM); | 0.61 (CM) | – | – | – | ✓ |
| 0.69 (CM) | ||||||
| Gandy et al. (2013) | 0.76 (LM); | 0.82 (LM) | – | – | ✓ | – |
| 0.65 (CM) | ||||||
| Li et al. (2013) | 0.65–0.73 | 0.52–0.66 | 0.58–0.69 | – | – | ✓ |
| Shutova (2013) | 0.68 | 0.66 | 0.67 | – | – | ✓ |
The results of Gedigian et al. (2006) should be interpreted with a reference to the performance of an all metaphor baseline attaining 0.92.
Turney et al. (2011) report results on the verb dataset in terms of F-score and on the adjective dataset in terms of accuracy. LM stands for linguistic metaphor and CM for conceptual metaphor.