5 papers · 1 filter
Retrieval and Augmentation of Domain Knowledge for Text-to-SQL Semantic Parsing
Manasi Patwardhan, Ayush Agarwal, Shabbirhussain Bhaisaheb +3
The performance of Large Language Models (LLMs) for translating Natural Language (NL) queries into SQL varies significantly across databases (DBs). NL queries are often expressed u…
AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research
Yilun Zhao, Weiyuan Chen, Zhijian Xu +5
We introduce AbGen, the first benchmark designed to evaluate the capabilities of LLMs in designing ablation studies for scientific research. AbGen consists of 1,500 expert-annotate…
Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers
Zhijian Xu, Yilun Zhao, Manasi Patwardhan +2
Peer review is fundamental to scientific research, but the growing volume of publications has intensified the challenges of this expertise-intensive process. While LLMs show promis…
DBRouting: Routing End User Queries to Databases for Answerability
Priyangshu Mandal, Manasi Patwardhan, Mayur Patidar +1
Enterprise level data is often distributed across multiple sources and identifying the correct set-of data-sources with relevant information for a knowledge request is a fundamenta…
Translating Across Cultures: LLMs for Intralingual Cultural Adaptation
Pushpdeep Singh, Mayur Patidar, Lovekesh Vig
LLMs are increasingly being deployed for multilingual applications and have demonstrated impressive translation capabilities between several low and high-resource languages. An asp…