2 citations · 8 across the 33 of their papers we have counts for
36 papers
Confidence Reasoning Graphs: Structured Confidence Estimation for LLM Agents
Brendan King, Farima Fatahi Bayat, Jean-Flavien Bussotti +2
When using an LLM agent in a consequential domain, making an informed decision about whether to trust its output or intervene requires calibrated confidence in the agent's success.…
MAWILE: Multi-Axis Workbench for Inspecting LLM Evaluators
Jackson Hassell, Farima Fatahi Bayat, Pouya Pezeshkpour +1
Large language model (LLM) judges provide a flexible and scalable method for evaluating model and agent outputs, but their verdicts can be sensitive to incidental changes in the ev…
ABSOL: Aggregated Bayesian Subsampling Orchestrated with LLMs
Jackson Hassell, Chen Shen, Estevam Hruschka
Large language models are increasingly used as natural-language interfaces to structured data, yet they remain unreliable when answers require consistent evidence conditioning, dep…
Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenance
Chen Shen, Estevam Hruschka
Lifelong LLM agents increasingly rely on external skill artifacts as one element for preserving and reusing capabilities over time. These skills (usually portable Markdown files su…
Hypotheses-Guided Self Distillation for Continual Personalization
EunJeong Hwang, Kushan Mitra, Dan Zhang +2
As people increasingly interact with LLM assistants in daily life, continually adapting to individual preferences has become essential for effective long-term interactions. However…
How to Steer Your Multi-Agent System: Human-LLM Collaborative Planning
Zeyu He, Hannah Kim, Dan Zhang +1
In orchestrated multi-agent systems, humans often struggle to manage plans due to their complexity and limited transparency. Existing approaches rely on outcome-level supervision,…
Reflective Prompt Tuning through Language Model Function-Calling
Farima Fatahi Bayat, Moin Aminnaseri, Pouya Pezeshkpour +1
Large language models (LLMs) have become increasingly capable of following instructions and complex reasoning, making prompting a flexible interface for adapting models without par…
Do Agents Need to Plan Step-by-Step? Rethinking Planning Horizon in Data-Centric Tool Calling
Naoki Otani, Nikita Bhutani, Hannah Kim +2
Explicit planning is a critical capability for LLM-based agents solving complex data-centric tasks, which require precise tool calling over external data sources. Existing strategi…
AutoPyVerifier: Learning Compact Executable Verifiers for Large Language Model Outputs
Pouya Pezeshkpour, Estevam Hruschka
Verification is becoming central to both reinforcement-learning-based training and inference-time control of large language models (LLMs). Yet current verifiers face a fundamental…
Blue Data Intelligence Layer: Streaming Data and Agents for Multi-source Multi-modal Data-Centric Applications
Moin Aminnaseri, Farima Fatahi Bayat, Nikita Bhutani +17
NL2SQL systems aim to address the growing need for natural language interaction with data. However, real-world information rarely maps to a single SQL query because (1) users expre…
Align Then Adapt: Label-Efficient Adapter Learning for Asymmetric Dense Retrieval
Seiji Maekawa, Moin Aminnaseri, Pouya Pezeshkpour +1
Dense retrieval systems increasingly face an asymmetry between complex instruction-like queries and relatively simple, static document collections. While stronger embedders can bet…
From Textual Columns to Query Plans: A Unified Relational-Semantic Execution Framework for Hybrid Query Processing
Nima Shahbazi, Seiji Maekawa, Nikita Bhutani +1
Real-world table question answering often involves hybrid schemas in which some query-relevant information is explicit in relational columns, while other attributes, predicates, or…
A Dynamic Self-Evolving Extraction System
Moin Amin-Naseri, Hannah Kim, Estevam Hruschka
The extraction of structured information from raw text is a fundamental component of many NLP applications, including document retrieval, ranking, and relevance estimation. High-qu…
From Task Solving to Robust Real-World Adaptation in LLM Agents
Pouya Pezeshkpour, Estevam Hruschka
Large language models are increasingly deployed as specialized agents that plan, call tools, and take actions over extended horizons. Yet many existing evaluations assume a "clean…
From Proof to Program: Characterizing Tool-Induced Reasoning Hallucinations in Large Language Models
Farima Fatahi Bayat, Pouya Pezeshkpour, Estevam Hruschka
Tool-augmented Language Models (TaLMs) can invoke external tools to solve problems beyond their parametric capacity. However, it remains unclear whether these tool-enabled gains re…
Verification-Aware Planning for Multi-Agent Systems
Tianyang Xu, Dan Zhang, Kushan Mitra +1
Large language model (LLM) agents are increasingly deployed to tackle complex tasks, often necessitating collaboration among multiple specialized agents. However, multi-agent colla…
Learning from Supervision with Semantic and Episodic Memory: A Reflective Approach to Agent Adaptation
Jackson Hassell, Dan Zhang, Hannah Kim +2
We investigate how agents built on pretrained large language models (LLMs) can learn target classification functions from labeled examples without parameter updates. While conventi…
AIPOM: Agent-aware Interactive Planning for Multi-Agent Systems
Hannah Kim, Kushan Mitra, Chen Shen +2
Large language models (LLMs) are being increasingly used for planning in orchestrated multi-agent systems. However, existing LLM-based approaches often fall short of human expectat…
Towards Reliable Benchmarking: A Contamination Free, Controllable Evaluation Framework for Multi-step LLM Function Calling
Seiji Maekawa, Jackson Hassell, Pouya Pezeshkpour +2
Existing benchmarks for tool-augmented language models (TaLMs) lack fine-grained control over task difficulty and remain vulnerable to data contamination. We present FuncBenchGen,…
RECAP: REwriting Conversations for Intent Understanding in Agentic Planning
Kushan Mitra, Dan Zhang, Hannah Kim +1
Understanding user intent is essential for effective planning in conversational assistants, particularly those powered by large language models (LLMs) coordinating multiple agents.…
Towards Probabilistic Question Answering Over Tabular Data
Chen Shen, Sajjadur Rahman, Estevam Hruschka
Progress in question answering (QA) over tabular data has enabled reliable factual retrieval from relational tables. However, many real-world questions are probabilistic, requiring…
Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict
Pouya Pezeshkpour, Moin Aminnaseri, Estevam Hruschka
Vision-language models (VLMs) have demonstrated impressive performance by effectively integrating visual and textual information to solve complex tasks. However, it is not clear ho…
Orchestrating Agents and Data for Enterprise: A Blueprint Architecture for Compound AI
Eser Kandogan, Nikita Bhutani, Dan Zhang +3
Large language models (LLMs) have gained significant interest in industry due to their impressive capabilities across a wide range of tasks. However, the widespread adoption of LLM…
Insight-RAG: Enhancing LLMs with Insight-Driven Augmentation
Pouya Pezeshkpour, Estevam Hruschka
Retrieval Augmented Generation (RAG) frameworks have shown significant promise in leveraging external knowledge to enhance the performance of large language models (LLMs). However,…