18 papers
ICMAPE: In-Context Multiagent Pure Exploration
Xinyi Hu, Alessio Russo, Aldo Pacchiano
In some multi-agent systems, the quantity to be optimized is not an externally specified reward but the information acquired about unknown properties of the environment as done in…
Future Information-Directed Sampling for Bayesian Nonstationary Bandits
Yichen Song, Alessio Russo, Aldo Pacchiano
Exploration--exploitation is a central trade-off in bandit learning. While classical algorithms such as upper confidence bound methods and Thompson Sampling effectively balance thi…
Non-Asymptotic Best Policy Identification Guarantees in Online Reinforcement Learning
Joseph Lazzaro, Alessio Russo, Aldo Pacchiano
In this work we study the Best Policy Identification (BPI) problem in online, tabular Reinforcement Learning. This is an active sequential hypothesis testing problem in which the l…
Bayesian Online Model Selection
Aida Afshar, Yuke Zhang, Aldo Pacchiano
Online model selection in Bayesian bandits raises a fundamental exploration challenge: When an environment instance is sampled from a prior distribution, how can we design an adapt…
In-Context Pure Exploration in Continuous Decision Spaces
Alessio Russo, Yin-Ching Lee, Ryan Welch +1
In active sequential testing, also termed pure exploration, a learner is tasked with the goal to adaptively acquire information so as to identify an unknown ground-truth hypothesis…
Principled Fine-tuning of LLMs from User-Edits: A Medley of Preference, Supervision, and Reward
Dipendra Misra, Aldo Pacchiano, Ta-Chung Chi +1
We study how to fine-tune LLMs using user-edit deployment data consisting of a set of context, an agent's response, and user edits. This deployment data is naturally generated by u…
Improved Training Mechanism for Reinforcement Learning via Online Model Selection
Aida Afshar, Aldo Pacchiano
We study the problem of online model selection in reinforcement learning, where the selector has access to a class of reinforcement learning agents and learns to adaptively select…
On the Hardness of Bandit Learning
Nataly Brukhim, Aldo Pacchiano, Miroslav Dudik +1
We study the task of bandit learning, also known as best-arm identification, under the assumption that the true reward function f belongs to a known, but arbitrary, function class…
Meet Me at the Arm: The Cooperative Multi-Armed Bandits Problem with Shareable Arms
Xinyi Hu, Aldo Pacchiano
We study the decentralized multi-player multi-armed bandits (MMAB) problem under a no-sensing setting, where each player receives only their own reward and obtains no information a…
In-Context Learning for Pure Exploration
Alessio Russo, Ryan Welch, Aldo Pacchiano
We study the problem active sequential hypothesis testing, also known as pure exploration: given a new task, the learner adaptively collects data from the environment to efficientl…
Pure Exploration with Feedback Graphs
Alessio Russo, Yichen Song, Aldo Pacchiano
We study the sample complexity of pure exploration in an online learning problem with a feedback graph. This graph dictates the feedback available to the learner, covering scenario…
Adaptive Exploration for Multi-Reward Multi-Policy Evaluation
Alessio Russo, Aldo Pacchiano
We study the policy evaluation problem in an online multi-reward multi-policy discounted setting, where multiple reward functions must be evaluated simultaneously for different pol…
ORSO: Accelerating Reward Design via Online Reward Selection and Policy Optimization
Chen Bo Calvin Zhang, Zhang-Wei Hong, Aldo Pacchiano +1
Reward shaping is critical in reinforcement learning (RL), particularly for complex tasks where sparse rewards can hinder learning. However, choosing effective shaping rewards from…
State-free Reinforcement Learning
Mingyu Chen, Aldo Pacchiano, Xuezhou Zhang
In this work, we study the \textit{state-free RL} problem, where the algorithm does not have the states information before interacting with the environment. Specifically, denote th…
Second Order Bounds for Contextual Bandits with Function Approximation
Aldo Pacchiano
Many works have developed no-regret algorithms for contextual bandits with function approximation, where the mean reward function over context-action pairs belongs to a function cl…
Learning Rate-Free Reinforcement Learning: A Case for Model Selection with Non-Stationary Objectives
Aida Afshar, Aldo Pacchiano
The performance of reinforcement learning (RL) algorithms is sensitive to the choice of hyperparameters, with the learning rate being particularly influential. RL algorithms fail t…
Provable Interactive Learning with Hindsight Instruction Feedback
Dipendra Misra, Aldo Pacchiano, Robert E. Schapire
We study interactive learning in a setting where the agent has to generate a response (e.g., an action or trajectory) given a context and an instruction. In contrast, to typical ap…
Multiple-policy Evaluation via Density Estimation
Yilei Chen, Aldo Pacchiano, Ioannis Ch. Paschalidis
We study the multiple-policy evaluation problem where we are given a set of policies and the goal is to evaluate their performance (expected total reward over a fixed horizon)…