State-of-the-art systems on CoNLL-2014 data set. The systems are divided into three categories: classifiers, MT, and combined. For each system, we also show the size of the annotated learner data used to train the system. Depending on the type of training data available, a specific approach should be preferred. In particular, classification systems can be trained from significantly less annotated data and can focus on specific errors, whereas MT-based systems do not have this capability. Best result in each category is in bold.
System type . | System name . | Annotated learner data . | F0.5 . | ||
---|---|---|---|---|---|
CoNLL . | Lang-8 . | CLC . | |||
1.2M . | 11-48M . | 29M . | |||
Classif. | Susanto, Phandi, and Ng (2014) | ✓ | 35.44 | ||
Rozovskaya and Roth (2016) | ✓ | 43.11 | |||
MT | Mizumoto and Matsumoto (2016) | ✓ | ✓ | 40.00 | |
Yuan and Briscoe (2016) | ✓ | ✓ | 39.90 | ||
Chollampatt, Taghipour, and Ng (2016) | ✓ | ✓ | 41.75 | ||
Hoang, Chollampatt, and Ng (2016) | ✓ | ✓ | 41.19 | ||
Rozovskaya and Roth (2016) | ✓ | 28.25 | |||
Rozovskaya and Roth (2016) | ✓ | ✓ | 39.48 | ||
Junczys-Dowmunt and Grundkiewicz (2016) | ✓ | ✓ | 49.49 | ||
Combined | Susanto, Phandi, and Ng (2014) | ✓ | ✓ | 39.39 | |
Rozovskaya and Roth (2016) | ✓ | ✓ | 47.40 |
System type . | System name . | Annotated learner data . | F0.5 . | ||
---|---|---|---|---|---|
CoNLL . | Lang-8 . | CLC . | |||
1.2M . | 11-48M . | 29M . | |||
Classif. | Susanto, Phandi, and Ng (2014) | ✓ | 35.44 | ||
Rozovskaya and Roth (2016) | ✓ | 43.11 | |||
MT | Mizumoto and Matsumoto (2016) | ✓ | ✓ | 40.00 | |
Yuan and Briscoe (2016) | ✓ | ✓ | 39.90 | ||
Chollampatt, Taghipour, and Ng (2016) | ✓ | ✓ | 41.75 | ||
Hoang, Chollampatt, and Ng (2016) | ✓ | ✓ | 41.19 | ||
Rozovskaya and Roth (2016) | ✓ | 28.25 | |||
Rozovskaya and Roth (2016) | ✓ | ✓ | 39.48 | ||
Junczys-Dowmunt and Grundkiewicz (2016) | ✓ | ✓ | 49.49 | ||
Combined | Susanto, Phandi, and Ng (2014) | ✓ | ✓ | 39.39 | |
Rozovskaya and Roth (2016) | ✓ | ✓ | 47.40 |