Evaluation results of the proposed systems and other state-of-the-art systems.
| System . | Dev . | Test . |
|---|---|---|
| Without pretrained language models | ||
| GrammarSQL (Lin et al. 2019) | 34.8% | 33.8% |
| EditSQL (Zhang et al. 2019) | 36.4% | 32.9% |
| IRNet (Guo et al. 2019) | 53.3% | 46.7% |
| RATSQL v2 (Wang et al. 2020) | 62.7% | 57.2% |
| RYANSQL (Ours) | 43.4% | − |
| With pretrained language models | ||
| RCSQL (Lee 2019) | 28.5% | 24.3% |
| EditSQL + BERT | 57.6% | 53.4% |
| IRNet + BERT | 61.9% | 54.7% |
| IRNet v2 + BERT | 63.9% | 55.0% |
| SLSQL + BERT (Lei et al. 2020) | 60.8% | 55.7% |
| RYANSQL + BERT (Ours) | 66.6% | 58.2% |
| RYANSQL v2 + BERT (Ours) | 70.6% | 60.6% |
| With DB content | ||
| Global-GNN (Bogin, Gardner, and Berant 2019) | 52.7% | 47.4% |
| IRNet++ + XLNet | 65.5% | 60.1% |
| RATSQL v3 + BERT | 69.7% | 65.6% |
| System . | Dev . | Test . |
|---|---|---|
| Without pretrained language models | ||
| GrammarSQL (Lin et al. 2019) | 34.8% | 33.8% |
| EditSQL (Zhang et al. 2019) | 36.4% | 32.9% |
| IRNet (Guo et al. 2019) | 53.3% | 46.7% |
| RATSQL v2 (Wang et al. 2020) | 62.7% | 57.2% |
| RYANSQL (Ours) | 43.4% | − |
| With pretrained language models | ||
| RCSQL (Lee 2019) | 28.5% | 24.3% |
| EditSQL + BERT | 57.6% | 53.4% |
| IRNet + BERT | 61.9% | 54.7% |
| IRNet v2 + BERT | 63.9% | 55.0% |
| SLSQL + BERT (Lei et al. 2020) | 60.8% | 55.7% |
| RYANSQL + BERT (Ours) | 66.6% | 58.2% |
| RYANSQL v2 + BERT (Ours) | 70.6% | 60.6% |
| With DB content | ||
| Global-GNN (Bogin, Gardner, and Berant 2019) | 52.7% | 47.4% |
| IRNet++ + XLNet | 65.5% | 60.1% |
| RATSQL v3 + BERT | 69.7% | 65.6% |