Identification systems: Evaluation set-up.
| System . | Annotated corpus . | Continuous text . | Human judgments . | Individual selected examples . | MML . | AMT . |
|---|---|---|---|---|---|---|
| Mason (2004) | – | – | ✓ | ✓ | ✓ | – |
| Birke and Sarkar (2006) | – | – | – | ✓ | – | – |
| Gedigian et al. (2006) | – | – | – | ✓ | – | – |
| Krishnakumaran and Zhu (2007) | – | – | – | ✓ | ✓ | – |
| Shutova et al. (2010) | – | ✓ | ✓ | – | – | – |
| Li and Sporleder (2010) | – | – | – | ✓ | – | – |
| Turney et al. (2011) | – | – | – | ✓ | – | – |
| Neuman et al. (2013) | – | – | – | ✓ | – | – |
| Dunn (2013) | ✓ | ✓ | – | – | – | – |
| Tsvetkov et al. (2013, 2014) | – | – | – | ✓ | – | – |
| Mohler et al. (2013) | – | – | – | ✓ | – | – |
| Hentz et al. (2013) | – | – | ✓ | – | – | ✓ |
| Hovy et al. (2013) | – | – | – | ✓ | – | ✓ |
| Wilks et al. (2013) | – | – | – | ✓ | – | – |
| Strzalkowski et al. (2013) | – | – | ✓ | – | – | ✓ |
| Shutova and Sun (2013) | – | ✓ | ✓ | – | – | – |
| Gandy et al. (2013) | – | – | – | ✓ | – | – |
| Li et al. (2013) | – | – | – | ✓ | ✓ | ✓ |
| Shutova (2013) | ✓ | ✓ | – | – | – | – |
| System . | Annotated corpus . | Continuous text . | Human judgments . | Individual selected examples . | MML . | AMT . |
|---|---|---|---|---|---|---|
| Mason (2004) | – | – | ✓ | ✓ | ✓ | – |
| Birke and Sarkar (2006) | – | – | – | ✓ | – | – |
| Gedigian et al. (2006) | – | – | – | ✓ | – | – |
| Krishnakumaran and Zhu (2007) | – | – | – | ✓ | ✓ | – |
| Shutova et al. (2010) | – | ✓ | ✓ | – | – | – |
| Li and Sporleder (2010) | – | – | – | ✓ | – | – |
| Turney et al. (2011) | – | – | – | ✓ | – | – |
| Neuman et al. (2013) | – | – | – | ✓ | – | – |
| Dunn (2013) | ✓ | ✓ | – | – | – | – |
| Tsvetkov et al. (2013, 2014) | – | – | – | ✓ | – | – |
| Mohler et al. (2013) | – | – | – | ✓ | – | – |
| Hentz et al. (2013) | – | – | ✓ | – | – | ✓ |
| Hovy et al. (2013) | – | – | – | ✓ | – | ✓ |
| Wilks et al. (2013) | – | – | – | ✓ | – | – |
| Strzalkowski et al. (2013) | – | – | ✓ | – | – | ✓ |
| Shutova and Sun (2013) | – | ✓ | ✓ | – | – | – |
| Gandy et al. (2013) | – | – | – | ✓ | – | – |
| Li et al. (2013) | – | – | – | ✓ | ✓ | ✓ |
| Shutova (2013) | ✓ | ✓ | – | – | – | – |