137 citations · 656 across the 54 of their papers we have counts for
55 papers
A Convex Framework for Confounding Robust Inference
Kei Ishikawa, Niao He, Takafumi Kanamori
We study policy evaluation of offline contextual bandits subject to unobserved confounders. Sensitivity analysis methods are commonly used to estimate the policy value under the wo…
On Imitation in Mean-field Games
Giorgia Ramponi, Pavel Kolev, Olivier Pietquin +3
We explore the problem of imitation learning (IL) in the context of mean-field games (MFGs), where the goal is to imitate the behavior of a population of agents following a Nash eq…
Provably Convergent Policy Optimization via Metric-aware Trust Region Methods
Jun Song, Niao He, Lijun Ding +1
Trust-region methods based on Kullback-Leibler divergence are pervasively used to stabilize policy optimization in reinforcement learning. In this paper, we exploit more flexible m…
Provably Learning Nash Policies in Constrained Markov Potential Games
Pragnya Alatur, Giorgia Ramponi, Niao He +1
Multi-agent reinforcement learning (MARL) addresses sequential decision-making problems with multiple agents, where each agent optimizes its own objective. In many real-world insta…
Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space
Anas Barakat, Ilyas Fatkhullin, Niao He
We consider the reinforcement learning (RL) problem with general utilities which consists in maximizing a function of the state-action occupancy measure. Beyond the standard cumula…
Cancellation-Free Regret Bounds for Lagrangian Approaches in Constrained Markov Decision Processes
Adrian Müller, Pragnya Alatur, Giorgia Ramponi +1
Constrained Markov Decision Processes (CMDPs) are one of the common ways to model safe reinforcement learning problems, where constraint functions model the safety objectives. Lagr…
Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods
Junchi Yang, Xiang Li, Ilyas Fatkhullin +1
The classical analysis of Stochastic Gradient Descent (SGD) with polynomially decaying stepsize relies on well-tuned depending on problem parameters such as…
Kernel Conditional Moment Constraints for Confounding Robust Inference
Kei Ishikawa, Niao He
We study policy evaluation of offline contextual bandits subject to unobserved confounders. Sensitivity analysis methods are commonly used to estimate the policy value under the wo…
Robust Knowledge Transfer in Tiered Reinforcement Learning
Jiawei Huang, Niao He
In this paper, we study the Tiered Reinforcement Learning setting, a parallel transfer learning framework, where the goal is to transfer knowledge from the low-tier (source) task t…
Stochastic Policy Gradient Methods: Improved Sample Complexity for Fisher-non-degenerate Policies
Ilyas Fatkhullin, Anas Barakat, Anastasia Kireeva +1
Recently, the impressive empirical success of policy gradient (PG) methods has catalyzed the development of their theoretical foundations. Despite the huge efforts directed at the…
Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Games
Batuhan Yardim, Semih Cayci, Matthieu Geist +1
Mean-field games have been used as a theoretical tool to obtain an approximate Nash equilibrium for symmetric and anonymous -player games. However, limiting applicability, exist…
Sharp Analysis of Stochastic Optimization under Global Kurdyka-Łojasiewicz Inequality
Ilyas Fatkhullin, Jalal Etesami, Niao He +1
We study the complexity of finding the global solution to stochastic nonconvex optimization when the objective function satisfies global Kurdyka-Lojasiewicz (KL) inequality and the…
TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization
Xiang Li, Junchi Yang, Niao He
Adaptive gradient methods have shown their ability to adjust the stepsizes on the fly in a parameter-agnostic manner, and empirically achieve faster convergence for solving minimiz…
Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm
Semih Cayci, Niao He, R. Srikant
Natural actor-critic (NAC) and its variants, equipped with the representation power of neural networks, have demonstrated impressive empirical success in solving Markov decision pr…
Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization
Junchi Yang, Xiang Li, Niao He
Adaptive algorithms like AdaGrad and AMSGrad are successful in nonconvex optimization owing to their parameter-agnostic ability -- requiring no a priori knowledge about problem-spe…
Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimization
Liang Zhang, Kiran Koshy Thekumparampil, Sewoong Oh +1
We study differentially private (DP) algorithms for smooth stochastic minimax optimization, with stochastic minimization as a byproduct. The holy grail of these settings is to guar…
Generalization Bounds of Nonconvex-(Strongly)-Concave Stochastic Minimax Optimization
Siqi Zhang, Yifan Hu, Liang Zhang +1
This paper takes an initial step to systematically investigate the generalization bounds of algorithms for solving nonconvex-(strongly)-concave (NC-SC/NC-C) stochastic minimax opti…
Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function
Saeed Masiha, Saber Salehkaleybar, Niao He +2
We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with which holds in a wide range o…
Momentum-Based Policy Gradient with Second-Order Information
Saber Salehkaleybar, Sadegh Khorasani, Negar Kiyavash +2
Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleratio…
Efficient Algorithms for A Class of Stochastic Hidden Convex Optimization and Its Applications in Network Revenue Management
Xin Chen, Niao He, Yifan Hu +1
We study a class of stochastic nonconvex optimization in the form of , i.e., is a composition of a convex function and…
Finite-Time Analysis of Natural Actor-Critic for POMDPs
Semih Cayci, Niao He, R. Srikant
We consider the reinforcement learning problem for partially observed Markov decision processes (POMDPs) with large or even countably infinite state spaces, where the controller ha…
Lifted Primal-Dual Method for Bilinearly Coupled Smooth Minimax Optimization
Kiran Koshy Thekumparampil, Niao He, Sewoong Oh
We study the bilinearly coupled minimax problem: , where and are both strongly convex smooth functions and admit first-order gra…
Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity
Junchi Yang, Antonio Orvieto, Aurelien Lucchi +1
Gradient descent ascent (GDA), the simplest single-loop algorithm for nonconvex minimax optimization, is widely used in practical applications such as generative adversarial networ…
Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation
Semih Cayci, Niao He, R. Srikant
Natural policy gradient (NPG) methods with entropy regularization achieve impressive empirical success in reinforcement learning problems with large state-action spaces. However, t…