3 citations · 4 across the 20 of their papers we have counts for
21 papers
Ask Less, Ask Better: Allocating User Attention in Proactive LLM Agents with Interruption Budgets
Xuanming Zhang, Amir H. Rezaeian, Miguel Ballesteros +6
LLM agents can ask clarifying questions to complete tasks, but answering each question demands users' time and effort. Existing evaluations typically measure this burden by countin…
MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversation
Jyotika Singh, Fang Tu, Miguel Ballesteros +6
Large language models (LLMs) suffer significant performance degradation when user instructions and context are distributed over multiple conversational turns, yet multi-turn (MT) i…
GSM-SEM: Benchmark and Framework for Generating Semantically Variant Augmentations
Jyotika Singh, Fang Tu, Aziza Mirsaidova +11
Benchmarks like GSM8K are popular measures of mathematical reasoning, but leaderboard gains can overstate true capability due to memorization of fixed test sets. Most robustness va…
JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents
Sandip Ghoshal, Anshul Mittal, Jyotika Singh +9
Large language model (LLM) agents augmented with external tools often struggle as number of tools grow large and become domain-specific. In such settings, ambiguous tool descriptio…
DiffuMask: Diffusion Language Model for Token-level Prompt Pruning
Caleb Zheng, Jyotika Singh, Fang Tu +6
In-Context Learning and Chain-of-Thought prompting improve reasoning in large language models (LLMs). These typically come at the cost of longer, more expensive prompts that may co…
Barriers to Discrete Reasoning with Transformers: A Survey Across Depth, Exactness, and Bandwidth
Michelle Yuan, Weiyi Sun, Amir H. Rezaeian +5
Transformers have become the foundational architecture for a broad spectrum of sequence modeling applications, underpinning state-of-the-art systems in natural language processing,…
Open Agent Specification (Agent Spec): A Unified Representation for AI Agents
Soufiane Amini, Yassine Benajiba, Cesare Bernardis +16
The proliferation of agent frameworks has led to fragmentation in how agents are defined, executed, and evaluated. Existing systems differ in their abstractions, data flow semantic…
Can LLMs Narrate Tabular Data? An Evaluation Framework for Natural Language Representations of Text-to-SQL System Outputs
Jyotika Singh, Weiyi Sun, Amit Agarwal +4
In modern industry systems like multi-turn chat agents, Text-to-SQL technology bridges natural language (NL) questions and database (DB) querying. The conversion of tabular DB resu…
Towards Long Context Hallucination Detection
Siyi Liu, Kishaloy Halder, Zheng Qi +6
Large Language Models (LLMs) have demonstrated remarkable performance across various tasks. However, they are prone to contextual hallucination, generating information that is eith…
MemInsight: Autonomous Memory Augmentation for LLM Agents
Rana Salama, Jason Cai, Michelle Yuan +4
Large language model (LLM) agents have evolved to intelligently process information, make decisions, and interact with users or tools. A key capability is the integration of long-t…
A Study on Leveraging Search and Self-Feedback for Agent Reasoning
Karthikeyan K, Michelle Yuan, Elman Mansimov +6
Recent works have demonstrated that incorporating search during inference can significantly improve reasoning capabilities of language agents. Some approaches may make use of the g…
Self-supervised Analogical Learning using Language Models
Ben Zhou, Sarthak Jain, Yi Zhang +4
Large language models have been shown to suffer from reasoning inconsistency issues. That is, they fail more in situations unfamiliar to the training data, even though exact or ver…
TReMu: Towards Neuro-Symbolic Temporal Reasoning for LLM-Agents with Memory in Multi-Session Dialogues
Yubin Ge, Salvatore Romeo, Jason Cai +4
Temporal reasoning in multi-session dialogues presents a significant challenge which has been under-studied in previous temporal reasoning benchmarks. To bridge this gap, we propos…
Rethinking LLM Uncertainty: A Multi-Agent Approach to Estimating Black-Box Model Uncertainty
Yu Feng, Phu Mon Htut, Zheng Qi +7
Quantifying uncertainty in black-box LLMs is vital for reliable responses and scalable oversight. Existing methods, which gauge a model's uncertainty through evaluating self-consis…
Inference time LLM alignment in single and multidomain preference spectrum
Sadat Shahriar, Zheng Qi, Nikolaos Pappas +5
Aligning Large Language Models (LLM) to address subjectivity and nuanced preference levels requires adequate flexibility and control, which can be a resource-intensive and time-con…
Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models
Qin Liu, Chao Shang, Ling Liu +7
The safety alignment ability of Vision-Language Models (VLMs) is prone to be degraded by the integration of the vision module compared to its LLM backbone. We investigate this phen…
Active Evaluation Acquisition for Efficient LLM Benchmarking
Yang Li, Jie Ma, Miguel Ballesteros +2
As large language models (LLMs) become increasingly versatile, numerous large scale benchmarks have been developed to thoroughly assess their capabilities. These benchmarks typical…
Open Domain Question Answering with Conflicting Contexts
Siyi Liu, Qiang Ning, Kishaloy Halder +8
Open domain question answering systems frequently rely on information retrieved from large collections of text (such as the Web) to answer questions. However, such collections of t…
General Purpose Verification for Chain of Thought Prompting
Robert Vacareanu, Anurag Pratik, Evangelia Spiliopoulou +6
Many of the recent capabilities demonstrated by Large Language Models (LLMs) arise primarily from their ability to exploit contextual information. In this paper, we explore ways to…
From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification
Fei Wang, Chao Shang, Sarthak Jain +6
User alignment is crucial for adapting general-purpose language models (LMs) to downstream tasks, but human annotations are often not available for all types of instructions, espec…
NewsQs: Multi-Source Question Generation for the Inquiring Mind
Alyssa Hwang, Kalpit Dixit, Miguel Ballesteros +5
We present NewsQs (news-cues), a dataset that provides question-answer pairs for multiple news documents. To create NewsQs, we augment a traditional multi-document summarization da…