ColloQL: Robust Cross-Domain Text-to-SQL Over Search Queries

Abstract

Translating natural language utterances to executable queries is a helpful technique in making the vast amount of data stored in relational databases accessible to a wider range of non-tech-savvy end users. Prior work in this area has largely focused on textual input that is linguistically correct and semantically unambiguous. However, real-world user queries are often succinct, colloquial, and noisy, resembling the input of a search engine. In this work, we introduce data augmentation techniques and a sampling-based content-aware BERT model (ColloQL) to achieve robust text-to-SQL modeling over natural language search (NLS) questions. Due to the lack of evaluation data, we curate a new dataset of NLS questions and demonstrate the efficacy of our approach. ColloQL's superior performance extends to well-formed text, achieving 84.9% (logical) and 90.7% (execution) accuracy on the WikiSQL dataset, making it, to the best of our knowledge, the highest performing model that does not use execution guided decoding.

arXiv commentIntEx-SemPar Workshop at EMNLP 2020, 12 pages, 3 figures

References in corpus (7)

• Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
• SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning
• Improving Text-to-SQL Evaluation Methodology
• A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization
• Hybrid Ranking Network for Text-to-SQL
• Content Enhanced BERT-based Text-to-SQL Generation
• Replace or Retrieve Keywords In Documents at Scale

Cited by in corpus (2)

• Towards Robustness of Text-to-SQL Models against Synonym Substitution
• KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers