137 citations · 656 across the 54 of their papers we have counts for
31 papers · 1 filter
On Imitation in Mean-field Games
Giorgia Ramponi, Pavel Kolev, Olivier Pietquin +3
We explore the problem of imitation learning (IL) in the context of mean-field games (MFGs), where the goal is to imitate the behavior of a population of agents following a Nash eq…
Provably Convergent Policy Optimization via Metric-aware Trust Region Methods
Jun Song, Niao He, Lijun Ding +1
Trust-region methods based on Kullback-Leibler divergence are pervasively used to stabilize policy optimization in reinforcement learning. In this paper, we exploit more flexible m…
Provably Learning Nash Policies in Constrained Markov Potential Games
Pragnya Alatur, Giorgia Ramponi, Niao He +1
Multi-agent reinforcement learning (MARL) addresses sequential decision-making problems with multiple agents, where each agent optimizes its own objective. In many real-world insta…
Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space
Anas Barakat, Ilyas Fatkhullin, Niao He
We consider the reinforcement learning (RL) problem with general utilities which consists in maximizing a function of the state-action occupancy measure. Beyond the standard cumula…
Cancellation-Free Regret Bounds for Lagrangian Approaches in Constrained Markov Decision Processes
Adrian Müller, Pragnya Alatur, Giorgia Ramponi +1
Constrained Markov Decision Processes (CMDPs) are one of the common ways to model safe reinforcement learning problems, where constraint functions model the safety objectives. Lagr…
Robust Knowledge Transfer in Tiered Reinforcement Learning
Jiawei Huang, Niao He
In this paper, we study the Tiered Reinforcement Learning setting, a parallel transfer learning framework, where the goal is to transfer knowledge from the low-tier (source) task t…
Stochastic Policy Gradient Methods: Improved Sample Complexity for Fisher-non-degenerate Policies
Ilyas Fatkhullin, Anas Barakat, Anastasia Kireeva +1
Recently, the impressive empirical success of policy gradient (PG) methods has catalyzed the development of their theoretical foundations. Despite the huge efforts directed at the…
Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm
Semih Cayci, Niao He, R. Srikant
Natural actor-critic (NAC) and its variants, equipped with the representation power of neural networks, have demonstrated impressive empirical success in solving Markov decision pr…
Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimization
Liang Zhang, Kiran Koshy Thekumparampil, Sewoong Oh +1
We study differentially private (DP) algorithms for smooth stochastic minimax optimization, with stochastic minimization as a byproduct. The holy grail of these settings is to guar…
Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function
Saeed Masiha, Saber Salehkaleybar, Niao He +2
We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with which holds in a wide range o…
Momentum-Based Policy Gradient with Second-Order Information
Saber Salehkaleybar, Sadegh Khorasani, Negar Kiyavash +2
Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleratio…
Finite-Time Analysis of Natural Actor-Critic for POMDPs
Semih Cayci, Niao He, R. Srikant
We consider the reinforcement learning problem for partially observed Markov decision processes (POMDPs) with large or even countably infinite state spaces, where the controller ha…
Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity
Junchi Yang, Antonio Orvieto, Aurelien Lucchi +1
Gradient descent ascent (GDA), the simplest single-loop algorithm for nonconvex minimax optimization, is widely used in practical applications such as generative adversarial networ…
Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation
Semih Cayci, Niao He, R. Srikant
Natural policy gradient (NPG) methods with entropy regularization achieve impressive empirical success in reinforcement learning problems with large state-action spaces. However, t…
Sample Complexity and Overparameterization Bounds for Temporal Difference Learning with Neural Network Approximation
Semih Cayci, Siddhartha Satpathi, Niao He +1
In this paper, we study the dynamics of temporal difference learning with neural network-based value function approximation over a general state space, namely, \emph{Neural TD lear…
The Mean-Squared Error of Double Q-Learning
Wentao Weng, Harsh Gupta, Niao He +2
In this paper, we establish a theoretical comparison between the asymptotic mean-squared error of Double Q-learning and Q-learning. Our result builds upon an analysis for linear st…
Periodic Q-Learning
Donghwan Lee, Niao He
The use of target networks is a common practice in deep reinforcement learning for stabilizing the training; however, theoretical understanding of this technique is still limited.…
Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents
Donghwan Lee, Niao He, Parameswaran Kamalaruban +1
This article reviews recent advances in multi-agent reinforcement learning algorithms for large-scale control systems and communication networks, which learn to communicate and coo…
Exponential Family Estimation via Adversarial Dynamics Embedding
Bo Dai, Zhen Liu, Hanjun Dai +4
We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural n…
Target-Based Temporal Difference Learning
Donghwan Lee, Niao He
The use of target networks has been a popular and key component of recent deep Q-learning algorithms for reinforcement learning, yet little is known from the theory side. In this w…
Quadratic Decomposable Submodular Function Minimization: Theory and Practice (Computation and Analysis of PageRank over Hypergraphs)
Pan Li, Niao He, Olgica Milenkovic
We introduce a new convex optimization problem, termed quadratic decomposable submodular function minimization (QDSFM), which allows to model a number of learning tasks on graphs a…
Kernel Exponential Family Estimation via Doubly Dual Embedding
Bo Dai, Hanjun Dai, Arthur Gretton +3
We investigate penalized maximum log-likelihood estimation for exponential family distributions whose natural parameter resides in a reproducing kernel Hilbert space. Key to our ap…
Quadratic Decomposable Submodular Function Minimization
Pan Li, Niao He, Olgica Milenkovic
We introduce a new convex optimization problem, termed quadratic decomposable submodular function minimization. The problem is closely related to decomposable submodular function m…
Boosting the Actor with Dual Critic
Bo Dai, Albert Shaw, Niao He +2
This paper proposes a new actor-critic-style algorithm called Dual Actor-Critic or Dual-AC. It is derived in a principled way from the Lagrangian dual form of the Bellman optimalit…