138 citations · 494 across the 31 of their papers we have counts for
32 papers
The Context Gathering Decision Process: A POMDP Framework for Agentic Search
Chinmaya Kausik, Adith Swaminathan, Nathan Kallus
Large Language Model (LLM) agents are deployed in complex environments -- such as massive codebases, enterprise databases, and conversational histories -- where the relevant state…
Understanding the Challenges in Iterative Generative Optimization with LLMs
Allen Nie, Xavier Daull, Zhiyi Kuang +10
Generative optimization uses large language models (LLMs) to iteratively improve artifacts (such as code, workflows or prompts) using execution feedback. It is a promising approach…
Reasoning about Reasoning: BAPO Bounds on Chain-of-Thought Token Complexity in LLMs
Kiran Tomlinson, Tobias Schnabel, Adith Swaminathan +1
Inference-time scaling via chain-of-thought (CoT) reasoning is a major driver of state-of-the-art LLM performance, but it comes with substantial latency and compute costs. We addre…
Formalizing Learning from Language Feedback with Provable Guarantees
Wanqiao Xu, Allen Nie, Ruijie Zheng +3
Interactively learning from observation and language feedback is an increasingly studied area driven by the emergence of large language model (LLM) agents. Despite impressive empir…
A Course Correction in Steerability Evaluation: Revealing Miscalibration and Side Effects in LLMs
Trenton Chang, Tobias Schnabel, Adith Swaminathan +1
Despite advances in large language models (LLMs) on reasoning and instruction-following tasks, it is unclear whether they can reliably produce outputs aligned with a variety of use…
Lost in Transmission: When and Why LLMs Fail to Reason Globally
Tobias Schnabel, Kiran Tomlinson, Adith Swaminathan +1
Despite their many successes, transformer-based large language models (LLMs) continue to struggle with tasks that require complex reasoning over large parts of their input. We argu…
Combining Open-box Simulation and Importance Sampling for Tuning Large-Scale Recommenders
Kaushal Paneri, Michael Munje, Kailash Singh Maurya +2
Growing scale of recommender systems require extensive tuning to respond to market dynamics and system changes. We address the challenge of tuning a large-scale ads recommendation…
How to Solve Contextual Goal-Oriented Problems with Offline Datasets?
Ying Fan, Jingling Li, Adith Swaminathan +2
We present a novel method, Contextual goal-Oriented Data Augmentation (CODA), which uses commonly available unlabeled trajectories and context-goal pairs to solve Contextual Goal-O…
The Importance of Directional Feedback for LLM-based Optimizers
Allen Nie, Ching-An Cheng, Andrey Kolobov +1
We study the potential of using large language models (LLMs) as an interactive optimizer for solving maximization problems in a text space using natural language and numerical feed…
Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs
Ching-An Cheng, Allen Nie, Adith Swaminathan
We study a class of optimization problems motivated by automating the design and update of AI systems like coding assistants, robots, and copilots. AutoDiff frameworks, like PyTorc…
On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbots
Christine Herlihy, Jennifer Neville, Tobias Schnabel +1
We explore the use of Large Language Model (LLM-based) chatbots to power recommender systems. We observe that the chatbots respond poorly when they encounter under-specified reques…
AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks
Jiacen Xu, Jack W. Stokes, Geoff McDonald +5
Large language models (LLMs) have demonstrated impressive results on natural language tasks, and security researchers are beginning to employ them in both offensive and defensive s…
LLF-Bench: Benchmark for Interactive Learning from Language Feedback
Ching-An Cheng, Andrey Kolobov, Dipendra Misra +2
We introduce a new benchmark, LLF-Bench (Learning from Language Feedback Benchmark; pronounced as "elf-bench"), to evaluate the ability of AI agents to interactively learn from nat…
Interactive Robot Learning from Verbal Correction
Huihan Liu, Alice Chen, Yuke Zhu +3
The ability to learn and refine behavior after deployment has become ever more important for robots as we design them to operate in unstructured environments like households. In th…
Hindsight Learning for MDPs with Exogenous Inputs
Sean R. Sinclair, Felipe Frujeri, Ching-An Cheng +6
Many resource management problems require sequential decision-making under uncertainty, where the only uncertainty affecting the decision outcomes are exogenous variables outside t…
Heuristic-Guided Reinforcement Learning
Ching-An Cheng, Andrey Kolobov, Adith Swaminathan
We provide a framework for accelerating reinforcement learning (RL) algorithms by heuristics constructed from domain knowledge or offline data. Tabula rasa RL algorithms require en…
Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learning
Bogdan Mazoure, Paul Mineiro, Pavithra Srinath +3
We study session-based recommendation scenarios where we want to recommend items to users during sequential interactions to improve their long-term utility. Optimizing a long-term…
Provably Good Batch Reinforcement Learning Without Great Exploration
Yao Liu, Adith Swaminathan, Alekh Agarwal +1
Batch reinforcement learning (RL) is important to apply RL algorithms to many high stakes tasks. Doing batch RL in a way that yields a reliable new policy in large domains is chall…
Improved Image Wasserstein Attacks and Defenses
Edward J. Hu, Adith Swaminathan, Hadi Salman +1
Robustness against image perturbations bounded by a ball have been well-studied in recent literature. Perturbations in the real-world, however, rarely exhibit the pixel in…
Working Memory Graphs
Ricky Loynd, Roland Fernandez, Asli Celikyilmaz +2
Transformers have increasingly outperformed gated RNNs in obtaining new state-of-the-art results on supervised tasks involving text sequences. Inspired by this trend, we study the…
Learning Calibratable Policies using Programmatic Style-Consistency
Eric Zhan, Albert Tseng, Yisong Yue +2
We study the problem of controllable generation of long-term sequential behaviors, where the goal is to calibrate to multiple behavior styles simultaneously. In contrast to the wel…
Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
Aditya Modi, Debadeepta Dey, Alekh Agarwal +4
Assemblies of modular subsystems are being pressed into service to perform sensing, reasoning, and decision making in high-stakes, time-critical tasks in such areas as transportati…
Multi-Preference Actor Critic
Ishan Durugkar, Matthew Hausknecht, Adith Swaminathan +1
Policy gradient algorithms typically combine discounted future rewards with an estimated value function, to compute the direction and magnitude of parameter updates. However, for m…
Off-Policy Policy Gradient with State Distribution Correction
Yao Liu, Adith Swaminathan, Alekh Agarwal +1
We study the problem of off-policy policy optimization in Markov decision processes, and develop a novel off-policy policy gradient method. Prior off-policy policy gradient approac…