2 citations · 2 across the 11 of their papers we have counts for
14 papers
RubricCraft: Where Solutions Diverge, Criteria Emerge
Hosein Hasanbeig, Priyanshu Gupta, Ashish Tiwari +2
Evaluating language agents on open-ended, artifact-producing tasks is difficult: model-based graders can be inconsistent, while exhaustive programmatic checks are often impractical…
Whose turn is it? Context-guided model routing for stateful agents
Anikesh Parashar, Avik Dutta, Vashu Chauhan +6
Model routing can reduce the cost of LLM applications by assigning simpler requests to cheaper models without sacrificing the quality of the response. However, most routers treat r…
Dude, Where's My State? Execution Information Requirements for Stateful Agents
Nikita Mehrotra, Ashish Tiwari, Priyanshu Gupta +1
Long-running agents must preserve information that later steps depend on. We introduce the Execution Information Requirement (EIR), a lower bound on the information that must remai…
LIFT: Last-Mile Fine-Tuning for Table Explicitation
Divij Khaitan, Ashish Tiwari
We propose last-mile fine-tuning, or Lift, a pipeline in which a pre-trained large language model extracts an initial table from unstructured clipboard text, and a fine-tuned small…
IndiMathBench: Autoformalizing Mathematical Reasoning Problems with a Human Touch
Param Biyani, Shashank Kirtania, Yasharth Bajpai +2
Reliable autoformalization remains challenging even in the era of large language models (LLMs). The scarcity of high-quality training data is a major bottleneck. Expert annotation…
ConDABench: Interactive Evaluation of Language Models for Data Analysis
Avik Dutta, Priyanshu Gupta, Hosein Hasanbeig +6
Real-world data analysis tasks often come with under-specified goals and unclean data. User interaction is necessary to understand and disambiguate a user's intent, and hence, esse…
TEN: Table Explicitization, Neurosymbolically
Nikita Mehrotra, Aayush Kumar, Sumit Gulwani +2
We present a neurosymbolic approach, TEN, for extracting tabular data from semistructured input text. This task is particularly challenging for text input that does not use special…
LLM-Guided Compositional Program Synthesis
Ruhma Khan, Sumit Gulwani, Vu Le +3
Program synthesis from input-output examples, also called programming by example (PBE), has had tremendous impact on automating end-user tasks. Large language models (LLMs) have th…
Ordered Semantically Diverse Sampling for Textual Data
Ashish Tiwari, Mukul Singh, Ananya Singha +1
The goal of diversity sampling is to select a representative subset of data in a way that maximizes information contained in the subset while keeping its cardinality small. We intr…
TableTalk: Scaffolding Spreadsheet Development with a Language Agent
Jenny T. Liang, Aayush Kumar, Yasharth Bajpai +7
Spreadsheet programming is challenging. Programmers use spreadsheet programming knowledge (e.g., formulas) and problem-solving skills to combine actions into complex tasks. Advance…
Conversational Challenges in AI-Powered Data Science: Obstacles, Needs, and Design Opportunities
Bhavya Chopra, Ananya Singha, Anna Fariha +4
Large Language Models (LLMs) are being increasingly employed in data science for tasks like data preprocessing and analytics. However, data scientists encounter substantial obstacl…
Augmented Embeddings for Custom Retrievals
Anirudh Khatry, Yasharth Bajpai, Priyanshu Gupta +2
Information retrieval involves selecting artifacts from a corpus that are most relevant to a given search query. The flavor of retrieval typically used in classical applications ca…
Rapidash: Efficient Constraint Discovery via Rapid Verification
Zifan Liu, Shaleen Deep, Anna Fariha +3
Denial Constraint (DC) is a well-established formalism that captures a wide range of integrity constraints commonly encountered, including candidate keys, functional dependencies,…
GrACE: Generation using Associated Code Edits
Priyanshu Gupta, Avishree Khare, Yasharth Bajpai +6
Developers expend a significant amount of time in editing code for a variety of reasons such as bug fixing or adding new features. Designing effective methods to predict code edits…