327 citations · 2.1k across the 124 of their papers we have counts for
89 papers · 1 filter
The Sample Complexity of Online Strategic Decision Making with Information Asymmetry and Knowledge Transportability
Jiachen Hu, Rui Ai, Han Zhong +4
Information asymmetry is a pervasive feature of multi-agent systems, especially evident in economics and social sciences. In these settings, agents tailor their actions based on pr…
Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learning
Chenjia Bai, Lingxiao Wang, Jianye Hao +4
Offline Reinforcement Learning (RL) has shown promising results in learning a task-specific policy from a fixed dataset. However, successful offline RL often relies heavily on the…
A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization
Yuchen Zhu, Yufeng Zhang, Zhaoran Wang +2
This paper studies minimax optimization problems defined over infinite-dimensional function classes of overparameterized two-layer neural networks. In particular, we consider the m…
Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning
Zihao Li, Boyi Liu, Zhuoran Yang +2
We study the Constrained Convex Markov Decision Process (MDP), where the goal is to minimize a convex functional of the visitation measure, subject to a convex constraint. Designin…
Posterior Sampling for Competitive RL: Function Approximation and Partial Observation
Shuang Qiu, Ziyu Dai, Han Zhong +3
This paper investigates posterior sampling algorithms for competitive reinforcement learning (RL) in the context of general function approximations. Focusing on zero-sum Markov gam…
Sample-Efficient Multi-Agent RL: An Optimization Perspective
Nuoya Xiong, Zhihan Liu, Zhaoran Wang +1
We study multi-agent reinforcement learning (MARL) for the general-sum Markov Games (MGs) under the general function approximation. In order to find the minimum assumption for samp…
Actions Speak What You Want: Provably Sample-Efficient Reinforcement Learning of the Quantal Stackelberg Equilibrium from Strategic Feedbacks
Siyu Chen, Mengdi Wang, Zhuoran Yang
We study reinforcement learning (RL) for learning a Quantal Stackelberg Equilibrium (QSE) in an episodic Markov game with a leader-follower structure. In specific, at the outset of…
Provably Efficient Representation Learning with Tractable Planning in Low-Rank POMDP
Jiacheng Guo, Zihao Li, Huazheng Wang +3
In this paper, we study representation learning in partially observable Markov Decision Processes (POMDPs), where the agent learns a decoder function that maps a series of high-dim…
A General Framework for Sequential Decision-Making under Adaptivity Constraints
Nuoya Xiong, Zhaoran Wang, Zhuoran Yang
We take the first step in studying general sequential decision-making under two adaptivity constraints: rare policy switch and batch learning. First, we provide a general class cal…
Provably Efficient Generalized Lagrangian Policy Optimization for Safe Multi-Agent Reinforcement Learning
Dongsheng Ding, Xiaohan Wei, Zhuoran Yang +2
We examine online safe multi-agent reinforcement learning using constrained Markov games in which agents compete by maximizing their expected total rewards under a constraint on ex…
Local Optimization Achieves Global Optimality in Multi-Agent Reinforcement Learning
Yulai Zhao, Zhuoran Yang, Zhaoran Wang +1
Policy optimization methods with function approximation are widely used in multi-agent reinforcement learning. However, it remains elusive how to design such algorithms with statis…
Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning
Haoran He, Chenjia Bai, Kang Xu +5
Diffusion models have demonstrated highly-expressive generative capabilities in vision and NLP. Recent studies in reinforcement learning (RL) have shown that diffusion models are a…
Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism
Zihao Li, Zhuoran Yang, Mengdi Wang
In this paper, we study offline Reinforcement Learning with Human Feedback (RLHF) where we aim to learn the human's underlying reward and the MDP's optimal policy from a set of tra…
Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Exploration
Zhihan Liu, Miao Lu, Wei Xiong +6
In online reinforcement learning (online RL), balancing exploration and exploitation is crucial for finding an optimal policy in a sample-efficient way. To achieve this, existing s…
Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization
Haoran Xu, Li Jiang, Jianxiong Li +4
Most offline reinforcement learning (RL) methods suffer from the trade-off between improving the policy to surpass the behavior policy and constraining the policy to limit the devi…
A Unified Framework of Policy Learning for Contextual Bandit with Confounding Bias and Missing Observations
Siyu Chen, Yitan Wang, Zhaoran Wang +1
We study the offline contextual bandit problem, where we aim to acquire an optimal policy using observational data. However, this data usually contains two deficiencies: (i) some v…
Learning to Incentivize Information Acquisition: Proper Scoring Rules Meet Principal-Agent Model
Siyu Chen, Jibang Wu, Yifan Wu +1
We study the incentivized information acquisition problem, where a principal hires an agent to gather information on her behalf. Such a problem is modeled as a Stackelberg game bet…
Offline Policy Optimization in RL with Variance Regularizaton
Riashat Islam, Samarth Sinha, Homanga Bharadhwaj +6
Learning policies from fixed offline datasets is a key challenge to scale up reinforcement learning (RL) algorithms towards practical applications. This is often because off-policy…
Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequality
Ying Jin, Zhimei Ren, Zhuoran Yang +1
This paper studies offline policy learning, which aims at utilizing observations collected a priori (from either fixed or adaptively evolving behavior policies) to learn an optimal…
GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond
Han Zhong, Wei Xiong, Sirui Zheng +4
We study sample efficient reinforcement learning (RL) under the general framework of interactive decision making, which includes Markov decision process (MDP), partially observable…
Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL
Fengzhuo Zhang, Boyi Liu, Kaixin Wang +3
The cooperative Multi-A gent R einforcement Learning (MARL) with permutation invariant agents framework has achieved tremendous empirical successes in real-world applications. Unfo…
A Reinforcement Learning Approach in Multi-Phase Second-Price Auction Design
Rui Ai, Boxiang Lyu, Zhaoran Wang +2
We study reserve price optimization in multi-phase second price auctions, where the seller's prior actions affect the bidders' later valuations through a Markov Decision Process (M…
Offline Reinforcement Learning with Instrumental Variables in Confounded Markov Decision Processes
Zuyue Fu, Zhengling Qi, Zhaoran Wang +3
We study the offline reinforcement learning (RL) in the face of unmeasured confounders. Due to the lack of online interaction with the environment, offline RL is facing the followi…
Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitions
Shuang Qiu, Xiaohan Wei, Jieping Ye +2
While single-agent policy optimization in a fixed environment has attracted a lot of research attention recently in the reinforcement learning community, much less is known theoret…