3 citations · 6 across the 22 of their papers we have counts for
23 papers
When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning
Luca Viano, Antoine Moulin, Audrey Huang +3
Imitation learning (IL)---training an agent to replicate expert behavior from demonstrations---underpins applications from robotics to language model training. Standard approaches…
Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation
Ziyad Sheebaelhamd, Luca Viano, Volkan Cevher +1
This work investigates multi-objective imitation learning: the problem of recovering policies that lie on the Pareto front given demonstrations from multiple Pareto-optimal experts…
Beyond Binary Preferences: A Principled Framework for Reward Modeling with Ordinal Feedback
Amirhossein Afsharrad, Ruida Zhou, Luca Viano +2
Reward modeling is crucial for aligning large language models with human preferences, yet current approaches lack a principled mathematical framework for leveraging ordinal prefere…
Multi-agent imitation learning with function approximation: Linear Markov games and beyond
Luca Viano, Till Freihaut, Emanuele Nevali +3
In this work, we present the first theoretical analysis of multi-agent imitation learning (MAIL) in linear Markov games where both the transition dynamics and each agent's reward f…
Provably avoiding over-optimization in Direct Preference Optimization without knowing the data distribution
Adam Barla, Emanuele Nevali, Luca Viano +1
We introduce PEPO (Pessimistic Ensemble based Preference Optimization), a single-step Direct Preference Optimization (DPO)-like algorithm to mitigate the well-known over-optimizati…
Direct Preference Optimization with Rating Information: Practical Algorithms and Provable Gains
Luca Viano, Ruida Zhou, Yifan Sun +4
The class of direct preference optimization (DPO) algorithms has emerged as a promising approach for solving the alignment problem in foundation models. These algorithms work with…
Rate optimal learning of equilibria from data
Till Freihaut, Luca Viano, Emanuele Nevali +3
We close open theoretical gaps in Multi-Agent Imitation Learning (MAIL) by characterizing the limits of non-interactive MAIL and presenting the first interactive algorithm with nea…
Inverse Q-Learning Done Right: Offline Imitation Learning in -Realizable MDPs
Antoine Moulin, Gergely Neu, Luca Viano
We study the problem of offline imitation learning in Markov decision processes (MDPs), where the goal is to learn a well-performing policy given a dataset of state-action pairs ge…
Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learning
Till Freihaut, Luca Viano, Volkan Cevher +2
This paper provides the first expert sample complexity characterization for learning a Nash equilibrium from expert data in Markov Games. We show that a new quantity named the sing…
IL-SOAR : Imitation Learning with Soft Optimistic Actor cRitic
Stefano Viel, Luca Viano, Volkan Cevher
This paper introduces the SOAR framework for imitation learning. SOAR is an algorithmic template that learns a policy from expert demonstrations with a primal dual style algorithm…
Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learning
Antoine Moulin, Gergely Neu, Luca Viano
We study the problem of reinforcement learning in infinite-horizon discounted linear Markov decision processes (MDPs), and propose the first computationally efficient algorithm ach…
Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guarantees
Yongtao Wu, Luca Viano, Yihang Chen +4
Reinforcement Learning from Human Feedback (RLHF) has been highly successful in aligning large language models with human preferences. While prevalent methods like DPO have demonst…
Best of Both Worlds: Regret Minimization versus Minimax Play
Adrian Müller, Jon Schneider, Stratis Skoulakis +2
In this paper, we investigate the existence of online learning algorithms with bandit feedback that simultaneously guarantee regret compared to a given comparator strategy,…
Imitation Learning in Discounted Linear MDPs without exploration assumptions
Luca Viano, Stratis Skoulakis, Volkan Cevher
We present a new algorithm for imitation learning in infinite horizon linear MDPs dubbed ILARL which greatly improves the bound on the number of trajectories that the learner needs…
Polynomial Convergence of Bandit No-Regret Dynamics in Congestion Games
Leello Dadi, Ioannis Panageas, Stratis Skoulakis +2
We introduce an online learning algorithm in the bandit feedback model that, once adopted by all agents of a congestion game, results in game-dynamics that converge to an -appro…
Semi Bandit Dynamics in Congestion Games: Convergence to Nash Equilibrium and No-Regret Guarantees
Ioannis Panageas, Stratis Skoulakis, Luca Viano +2
In this work, we introduce a new variant of online gradient descent, which provably converges to Nash Equilibria and simultaneously attains sublinear regret for the class of conges…
What can online reinforcement learning with function approximation benefit from general coverage conditions?
Fanghui Liu, Luca Viano, Volkan Cevher
In online reinforcement learning (RL), instead of employing standard structural assumptions on Markov decision processes (MDPs), using a certain coverage condition (original from o…
Understanding Deep Neural Function Approximation in Reinforcement Learning via -Greedy Exploration
Fanghui Liu, Luca Viano, Volkan Cevher
This paper provides a theoretical study of deep neural function approximation in reinforcement learning (RL) with the -greedy exploration under the online setting. This problem…
Identifiability and generalizability from multiple experts in Inverse Reinforcement Learning
Paul Rolland, Luca Viano, Norman Schuerhoff +2
While Reinforcement Learning (RL) aims to train an agent from a reward function in a given environment, Inverse Reinforcement Learning (IRL) seeks to recover the reward function fr…
Proximal Point Imitation Learning
Luca Viano, Angeliki Kamoutsi, Gergely Neu +2
This work develops new algorithms with rigorous efficiency guarantees for infinite horizon imitation learning (IL) with linear function approximation without restrictive coherence…
Robust Learning from Observation with Model Misspecification
Luca Viano, Yu-Ting Huang, Parameswaran Kamalaruban +3
Imitation learning (IL) is a popular paradigm for training policies in robotic systems when specifying the reward function is difficult. However, despite the success of IL algorith…
Neural NID Rules
Luca Viano, Johanni Brea
Abstract object properties and their relations are deeply rooted in human common sense, allowing people to predict the dynamics of the world even in situations that are novel but g…
Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch
Luca Viano, Yu-Ting Huang, Parameswaran Kamalaruban +2
We study the inverse reinforcement learning (IRL) problem under a transition dynamics mismatch between the expert and the learner. Specifically, we consider the Maximum Causal Entr…