50 citations · 275 across the 56 of their papers we have counts for
58 papers
Experiment Planning with Function Approximation
Aldo Pacchiano, Jonathan N. Lee, Emma Brunskill
We study the problem of experiment planning with function approximation in contextual bandit problems. In settings where there is a significant overhead to deploying adaptive algor…
Contextual Bandits with Stage-wise Constraints
Aldo Pacchiano, Mohammad Ghavamzadeh, Peter Bartlett
We study contextual bandits in the presence of a stage-wise constraint when the constraint must be satisfied both with high probability and in expectation. We start with the linear…
Unbiased Decisions Reduce Regret: Adversarial Domain Adaptation for the Bank Loan Problem
Elena Gal, Shaun Singh, Aldo Pacchiano +3
In many real world settings binary classification decisions are made based on limited data in near real-time, e.g. when assessing a loan application. We focus on a class of these p…
Anytime Model Selection in Linear Bandits
Parnian Kassraie, Nicolas Emmenegger, Andreas Krause +1
Model selection in the context of bandit optimization is a challenging problem, as it requires balancing exploration and exploitation not only for action selection, but also for mo…
Supervised Pretraining Can Learn In-Context Reinforcement Learning
Jonathan N. Lee, Annie Xie, Aldo Pacchiano +4
Large transformer models trained on diverse datasets have shown a remarkable ability to learn in-context, achieving high few-shot performance on tasks they were not explicitly trai…
A Unified Model and Dimension for Interactive Estimation
Nataly Brukhim, Miroslav Dudik, Aldo Pacchiano +1
We study an abstract framework for interactive learning called interactive estimation in which the goal is to estimate a target from its "similarity'' to points queried by the lear…
Data-Driven Online Model Selection With Regret Guarantees
Aldo Pacchiano, Christoph Dann, Claudio Gentile
We consider model selection for sequential decision making in stochastic environments with bandit feedback, where a meta-learner has at its disposal a pool of base learners, and de…
Improving Offline RL by Blending Heuristics
Sinong Geng, Aldo Pacchiano, Andrey Kolobov +1
We propose Heuristic Blending (HUBL), a simple performance-improving technique for a broad class of offline RL algorithms based on value bootstrapping. HUBL modifies the Bellman op…
Estimating Optimal Policy Value in General Linear Contextual Bandits
Jonathan N. Lee, Weihao Kong, Aldo Pacchiano +2
In many bandit problems, the maximal reward achievable by a policy is often unknown in advance. We consider the problem of estimating the optimal policy value in the sublinear data…
Transfer RL via the Undo Maps Formalism
Abhi Gupta, Ted Moskovitz, David Alvarez-Melis +1
Transferring knowledge across domains is one of the most fundamental problems in machine learning, but doing so effectively in the context of reinforcement learning remains largely…
Leveraging Offline Data in Online Reinforcement Learning
Andrew Wagenmaker, Aldo Pacchiano
Two central paradigms have emerged in the reinforcement learning (RL) community: online RL and offline RL. In the online RL setting, the agent has no prior knowledge of the environ…
Learning General World Models in a Handful of Reward-Free Deployments
Yingchen Xu, Jack Parker-Holder, Aldo Pacchiano +5
Building generally capable agents is a grand challenge for deep reinforcement learning (RL). To approach this challenge practically, we outline two key desiderata: 1) to facilitate…
Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexity
Abhishek Gupta, Aldo Pacchiano, Yuexiang Zhai +2
Reinforcement learning provides an automated framework for learning behaviors from high-level reward specifications, but in practice the choice of reward function can be crucial fo…
Neural Design for Genetic Perturbation Experiments
Aldo Pacchiano, Drausin Wulsin, Robert A. Barton +1
The problem of how to genetically modify cells in order to maximize a certain cellular phenotype has taken center stage in drug development over the last few years (with, for examp…
Best of Both Worlds Model Selection
Aldo Pacchiano, Christoph Dann, Claudio Gentile
We study the problem of model selection in bandit scenarios in the presence of nested policy classes, with the goal of obtaining simultaneous adversarial and stochastic ("best of b…
Joint Representation Training in Sequential Tasks with Shared Structure
Aldo Pacchiano, Ofir Nachum, Nilseh Tripuraneni +1
Classical theory in reinforcement learning (RL) predominantly focuses on the single task setting, where an agent learns to solve a task through trial-and-error experience, given ac…
Online Nonsubmodular Minimization with Delayed Costs: From Full Information to Bandit Feedback
Tianyi Lin, Aldo Pacchiano, Yaodong Yu +1
Motivated by applications to online learning in sparse estimation and Bayesian optimization, we consider the problem of online unconstrained nonsubmodular minimization with delayed…
Meta Learning MDPs with Linear Transition Models
Robert Müller, Aldo Pacchiano
We study meta-learning in Markov Decision Processes (MDP) with linear transition models in the undiscounted episodic setting. Under a task sharedness metric based on model proximit…
Neural Pseudo-Label Optimism for the Bank Loan Problem
Aldo Pacchiano, Shaun Singh, Edward Chou +2
We study a class of classification problems best exemplified by the \emph{bank loan} problem, where a lender decides whether or not to issue a loan. The lender only observes whethe…
An Instance-Dependent Analysis for the Cooperative Multi-Player Multi-Armed Bandit
Aldo Pacchiano, Peter Bartlett, Michael I. Jordan
We study the problem of information sharing and cooperation in Multi-Player Multi-Armed bandits. We propose the first algorithm that achieves logarithmic regret for this problem wh…
Dueling RL: Reinforcement Learning with Trajectory Preferences
Aldo Pacchiano, Aadirupa Saha, Jonathan Lee
We consider the problem of preference based reinforcement learning (PbRL), where, unlike traditional reinforcement learning, an agent receives feedback only in terms of a 1 bit (0/…
Towards an Understanding of Default Policies in Multitask Policy Optimization
Ted Moskovitz, Michael Arbel, Jack Parker-Holder +1
Much of the recent success of deep reinforcement learning has been driven by regularized policy optimization (RPO) algorithms with strong performance across multiple domains. In th…
Reinforcement Learning in Linear MDPs: Constant Regret and Representation Selection
Matteo Papini, Andrea Tirinzoni, Aldo Pacchiano +3
We study the role of the representation of state-action value functions in regret minimization in finite-horizon Markov Decision Processes (MDPs) with linear structure. We first de…
Sample Efficient Reinforcement Learning In Continuous State Spaces: A Perspective Beyond Linearity
Dhruv Malik, Aldo Pacchiano, Vishwak Srinivasan +1
Reinforcement learning (RL) is empirically successful in complex nonlinear Markov decision processes (MDPs) with continuous state spaces. By contrast, the majority of theoretical R…