Query (Q) and Interaction (I) accuracy for SParC and CoSQL. We report results on the development (D) and test (T) sets. Sparc-DI is our domain-independent split of SparC. HE is a hierarchical encoder and Mem is the proposed memory-based context encoder. LSTM is a vanilla decoder, SnipCopy copies SQL segments from the previous query, and Grammar refers to a decoder which outputs a sequence of grammar rules rather than tokens. Table cells are filled with—whenever results are not available.
| Model . | Enc-Dec . | CoSQL(D) . | CoSQL(T) . | SparC(D) . | SparC(T) . | SparC-DI(T) . | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Q . | I . | Q . | I . | Q . | I . | Q . | I . | Q . | I . | ||
| CDS2S | HE-LSTM | 13.8 | 2.1 | 13.9 | 2.6 | 21.9 | 8.1 | 23.2 | 7.5 | 39.5 | 20.1 |
| CDS2S | HE-SnipCopy | 12.3 | 2.1 | — | — | 21.7 | 9.5 | 20.3 | 8.1 | 38.7 | 24 |
| Liu et al. (2020) | HE-Grammar | 33.5 | 9.6 | — | — | 41.8 | 20.6 | — | — | 57.1 | 35.3 |
| MemCE+CDS2S | Mem-LSTM | 13.4 | 3.4 | — | — | 21.2 | 8.8 | — | — | 41.3 | 22.9 |
| MemCE+CDS2S | Mem-SnipCopy | 13.1 | 2.7 | — | — | 21.4 | 10.9 | — | — | 41.5 | 26.7 |
| MemCE+Liu et al. (2020) | Mem-Grammar | 32.8 | 10.6 | 28.4 | 6.2 | 42.4 | 21.1 | 40.3 | 16.7 | 55.7 | 36.3 |
| Model . | Enc-Dec . | CoSQL(D) . | CoSQL(T) . | SparC(D) . | SparC(T) . | SparC-DI(T) . | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Q . | I . | Q . | I . | Q . | I . | Q . | I . | Q . | I . | ||
| CDS2S | HE-LSTM | 13.8 | 2.1 | 13.9 | 2.6 | 21.9 | 8.1 | 23.2 | 7.5 | 39.5 | 20.1 |
| CDS2S | HE-SnipCopy | 12.3 | 2.1 | — | — | 21.7 | 9.5 | 20.3 | 8.1 | 38.7 | 24 |
| Liu et al. (2020) | HE-Grammar | 33.5 | 9.6 | — | — | 41.8 | 20.6 | — | — | 57.1 | 35.3 |
| MemCE+CDS2S | Mem-LSTM | 13.4 | 3.4 | — | — | 21.2 | 8.8 | — | — | 41.3 | 22.9 |
| MemCE+CDS2S | Mem-SnipCopy | 13.1 | 2.7 | — | — | 21.4 | 10.9 | — | — | 41.5 | 26.7 |
| MemCE+Liu et al. (2020) | Mem-Grammar | 32.8 | 10.6 | 28.4 | 6.2 | 42.4 | 21.1 | 40.3 | 16.7 | 55.7 | 36.3 |