3 citations · 9 across the 10 of their papers we have counts for
8 papers · 1 filter
On the Convergence of Single-Timescale Actor-Critic
Navdeep Kumar, Priyank Agrawal, Giorgia Ramponi +2
We analyze the global convergence of the single-timescale actor-critic (AC) algorithm for the infinite-horizon discounted Markov Decision Processes (MDPs) with finite state spaces.…
On the Global Convergence of Policy Gradient in Average Reward Markov Decision Processes
Navdeep Kumar, Yashaswini Murthy, Itai Shufaro +3
We present the first finite time global convergence analysis of policy gradient in the context of infinite horizon average reward Markov decision processes (MDPs). Specifically, we…
Solving Non-Rectangular Reward-Robust MDPs via Frequency Regularization
Uri Gadot, Esther Derman, Navdeep Kumar +3
In robust Markov decision processes (RMDPs), it is assumed that the reward and the transition dynamics lie in a given uncertainty set. By targeting maximal return under the most ad…
Bring Your Own (Non-Robust) Algorithm to Solve Robust MDPs by Estimating The Worst Kernel
Kaixin Wang, Uri Gadot, Navdeep Kumar +2
Robust Markov Decision Processes (RMDPs) provide a framework for sequential decision-making that is robust to perturbations on the transition kernel. However, current RMDP methods…
An Efficient Solution to s-Rectangular Robust Markov Decision Processes
Navdeep Kumar, Kfir Levy, Kaixin Wang +1
We present an efficient robust value iteration for \texttt{s}-rectangular robust Markov Decision Processes (MDPs) with a time complexity comparable to standard (non-robust) MDPs wh…
Policy Gradient for Rectangular Robust Markov Decision Processes
Navdeep Kumar, Esther Derman, Matthieu Geist +2
Policy gradient methods have become a standard for training reinforcement learning agents in a scalable and efficient manner. However, they do not account for transition uncertaint…
Policy Gradient for Reinforcement Learning with General Utilities
Navdeep Kumar, Kaixin Wang, Kfir Levy +1
In Reinforcement Learning (RL), the goal of agents is to discover an optimal policy that maximizes the expected cumulative rewards. This objective may also be viewed as finding a p…
The Geometry of Robust Value Functions
Kaixin Wang, Navdeep Kumar, Kuangqi Zhou +3
The space of value functions is a fundamental concept in reinforcement learning. Characterizing its geometric properties may provide insights for optimization and representation. E…