13 papers
Routing End User Queries to Enterprise Databases
Saikrishna Sudarshan, Tanay Kulkarni, Manasi Patwardhan +3
We address the task of routing natural language queries in multi-database enterprise environments. We construct realistic benchmarks by extending existing NL-to-SQL datasets. Our s…
Refine and Align: Confidence Calibration through Multi-Agent Interaction in VQA
Ayush Pandey, Jai Bardhan, Ishita Jain +3
In the context of Visual Question Answering (VQA) and Agentic AI, calibration refers to how closely an AI system's confidence in its answers reflects their actual correctness. This…
Retrieval and Augmentation of Domain Knowledge for Text-to-SQL Semantic Parsing
Manasi Patwardhan, Ayush Agarwal, Shabbirhussain Bhaisaheb +3
The performance of Large Language Models (LLMs) for translating Natural Language (NL) queries into SQL varies significantly across databases (DBs). NL queries are often expressed u…
SemSR: Semantics aware robust Session-based Recommendations
Jyoti Narwariya, Priyanka Gupta, Muskan Gupta +2
Session-based recommendation (SR) models aim to recommend items to anonymous users based on their behavior during the current session. While various SR models in the literature uti…
AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research
Yilun Zhao, Weiyuan Chen, Zhijian Xu +5
We introduce AbGen, the first benchmark designed to evaluate the capabilities of LLMs in designing ablation studies for scientific research. AbGen consists of 1,500 expert-annotate…
Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers
Zhijian Xu, Yilun Zhao, Manasi Patwardhan +2
Peer review is fundamental to scientific research, but the growing volume of publications has intensified the challenges of this expertise-intensive process. While LLMs show promis…