Model accuracy on the ATIS dataset. HE is a hierarchical interaction encoder, while Mem is the proposed memory-based encoder. LSTM are vanilla encoder/decoder models, while SnipCopy copies SQL segments from the previous query and EditBased adopts a query editing mechanism.
| Model . | Enc-Dec . | Dev Set . | Test Set . | ||||
|---|---|---|---|---|---|---|---|
| . | Denotation . | . | Denotation . | ||||
| Query . | Relaxed . | Strict . | Query . | Relaxed . | Strict . | ||
| Seq2Seq | LSTM-LSTM | 28.7 | 48.8 | 43.2 | 35.7 | 56.4 | 53.8 |
| Seq2Seq+Concat | LSTM-LSTM | 35.1 | 59.4 | 56.7 | 42.2 | 66.6 | 65.8 |
| Suhr et al. (2018) | HE-LSTM | 36.0 | 59.5 | 58.3 | — | — | — |
| Suhr et al. (2018) | HE-SnipCopy | 37.5 | 63.0 | 62.5 | 43.6 | 69.3 | 69.2 |
| Zhang et al. (2019) | HE-EditBased | 36.2 | 60.5 | 60.0 | 43.9 | 68.5 | 68.1 |
| Lin et al. (2019) | LSTM-Grammar | 39.1 | — | 65.8 | 44.1 | — | 73.7 |
| MemCE | Mem-LSTM | 40.2 | 63.6 | 61.2 | 47.0 | 70.1 | 68.9 |
| MemCE | Mem-SnipCopy | 39.1 | 65.5 | 65.2 | 45.3 | 70.2 | 69.8 |
| Model . | Enc-Dec . | Dev Set . | Test Set . | ||||
|---|---|---|---|---|---|---|---|
| . | Denotation . | . | Denotation . | ||||
| Query . | Relaxed . | Strict . | Query . | Relaxed . | Strict . | ||
| Seq2Seq | LSTM-LSTM | 28.7 | 48.8 | 43.2 | 35.7 | 56.4 | 53.8 |
| Seq2Seq+Concat | LSTM-LSTM | 35.1 | 59.4 | 56.7 | 42.2 | 66.6 | 65.8 |
| Suhr et al. (2018) | HE-LSTM | 36.0 | 59.5 | 58.3 | — | — | — |
| Suhr et al. (2018) | HE-SnipCopy | 37.5 | 63.0 | 62.5 | 43.6 | 69.3 | 69.2 |
| Zhang et al. (2019) | HE-EditBased | 36.2 | 60.5 | 60.0 | 43.9 | 68.5 | 68.1 |
| Lin et al. (2019) | LSTM-Grammar | 39.1 | — | 65.8 | 44.1 | — | 73.7 |
| MemCE | Mem-LSTM | 40.2 | 63.6 | 61.2 | 47.0 | 70.1 | 68.9 |
| MemCE | Mem-SnipCopy | 39.1 | 65.5 | 65.2 | 45.3 | 70.2 | 69.8 |