10 citations · 30 across the 8 of their papers we have counts for
8 papers · 1 filter
On Synthetic Data Strategies for Domain-Specific Generative Retrieval
Haoyang Wen, Jiang Guo, Yi Zhang +2
This paper investigates synthetic data generation strategies in developing generative retrieval models for domain-specific corpora, thereby addressing the scalability challenges in…
PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queries
Mingwen Dong, Nischal Ashok Kumar, Yiqun Hu +9
Previous text-to-SQL datasets and systems have primarily focused on user questions with clear intentions that can be answered. However, real user questions can often be ambiguous w…
RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation
Dongyu Ru, Lin Qiu, Xiangkun Hu +15
Despite Retrieval-Augmented Generation (RAG) showing promising capability in leveraging external knowledge, a comprehensive evaluation of RAG systems is still challenging due to th…
Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall
Jiaqing Yuan, Lin Pan, Chung-Wei Hang +5
Large language models (LLMs) have shown remarkable performance on a variety of NLP tasks, and are being rapidly adopted in a wide range of use cases. It is therefore of vital impor…
UNITE: A Unified Benchmark for Text-to-SQL Evaluation
Wuwei Lan, Zhiguo Wang, Anuj Chauhan +15
A practical text-to-SQL system should generalize well on a wide variety of natural language questions, unseen database schemas, and novel SQL query structures. To comprehensively e…
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness
Shuaichen Chang, Jun Wang, Mingwen Dong +13
Neural text-to-SQL models have achieved remarkable performance in translating natural language questions into SQL queries. However, recent studies reveal that text-to-SQL models ar…
Importance of Synthesizing High-quality Data for Text-to-SQL Parsing
Yiyun Zhao, Jiarong Jiang, Yiqun Hu +14
Recently, there has been increasing interest in synthesizing data to improve downstream text-to-SQL tasks. In this paper, we first examined the existing synthesized datasets and di…
Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding
Jun Wang, Patrick Ng, Alexander Hanbo Li +5
Most recent research on Text-to-SQL semantic parsing relies on either parser itself or simple heuristic based approach to understand natural language query (NLQ). When synthesizing…