6 papers
PLSQLBench: Benchmarking LLM Systems for Executable Procedural Database Programming
Marianne Menglin Liu, Leonid Boytsov, Daniel W. Peterson +13
We present PLSQLBench, to our knowledge the first benchmark for evaluating whether LLMs can write executable PL/SQL programs, with correctness measured through execution-based test…
SOMA-SQL: Resolving Multi-Source Ambiguity in NL-to-SQL via Synthetic Log and Execution Probing
Sai Ashish Somayajula, Marianne Menglin Liu, Chuan Lei +9
Natural language interfaces to databases aim to translate user questions into executable SQL, yet remain brittle in real-world settings where questions are underspecified and schem…
JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents
Sandip Ghoshal, Anshul Mittal, Jyotika Singh +9
Large language model (LLM) agents augmented with external tools often struggle as number of tools grow large and become domain-specific. In such settings, ambiguous tool descriptio…
DiffuMask: Diffusion Language Model for Token-level Prompt Pruning
Caleb Zheng, Jyotika Singh, Fang Tu +6
In-Context Learning and Chain-of-Thought prompting improve reasoning in large language models (LLMs). These typically come at the cost of longer, more expensive prompts that may co…
Routesplain: Towards Faithful and Intervenable Routing for Software-related Tasks
Adam Štorek, Vikas Upadhyay, Marianne Menglin Liu +6
LLMs now tackle a wide range of software-related tasks, yet we show that their performance varies markedly both across and within these tasks. Routing user queries to the appropria…
LAD-RAG: Layout-aware Dynamic RAG for Visually-Rich Document Understanding
Zhivar Sourati, Zheng Wang, Marianne Menglin Liu +8
Question answering over visually rich documents (VRDs) requires reasoning not only over isolated content but also over documents' structural organization and cross-page dependencie…