3 citations · 9 across the 10 of their papers we have counts for
10 papers
Dual Formulation for Non-Rectangular Lp Robust Markov Decision Processes
Navdeep Kumar, Adarsh Gupta, Maxence Mohamed Elfatihi +3
We study robust Markov decision processes (RMDPs) with non-rectangular uncertainty sets, which capture interdependencies across states unlike traditional rectangular models. While…
On the Convergence of Single-Timescale Actor-Critic
Navdeep Kumar, Priyank Agrawal, Giorgia Ramponi +2
We analyze the global convergence of the single-timescale actor-critic (AC) algorithm for the infinite-horizon discounted Markov Decision Processes (MDPs) with finite state spaces.…
On the Global Convergence of Policy Gradient in Average Reward Markov Decision Processes
Navdeep Kumar, Yashaswini Murthy, Itai Shufaro +3
We present the first finite time global convergence analysis of policy gradient in the context of infinite horizon average reward Markov decision processes (MDPs). Specifically, we…
Solving Non-Rectangular Reward-Robust MDPs via Frequency Regularization
Uri Gadot, Esther Derman, Navdeep Kumar +3
In robust Markov decision processes (RMDPs), it is assumed that the reward and the transition dynamics lie in a given uncertainty set. By targeting maximal return under the most ad…
Bring Your Own (Non-Robust) Algorithm to Solve Robust MDPs by Estimating The Worst Kernel
Kaixin Wang, Uri Gadot, Navdeep Kumar +2
Robust Markov Decision Processes (RMDPs) provide a framework for sequential decision-making that is robust to perturbations on the transition kernel. However, current RMDP methods…
An Efficient Solution to s-Rectangular Robust Markov Decision Processes
Navdeep Kumar, Kfir Levy, Kaixin Wang +1
We present an efficient robust value iteration for \texttt{s}-rectangular robust Markov Decision Processes (MDPs) with a time complexity comparable to standard (non-robust) MDPs wh…
Policy Gradient for Rectangular Robust Markov Decision Processes
Navdeep Kumar, Esther Derman, Matthieu Geist +2
Policy gradient methods have become a standard for training reinforcement learning agents in a scalable and efficient manner. However, they do not account for transition uncertaint…
Policy Gradient for Reinforcement Learning with General Utilities
Navdeep Kumar, Kaixin Wang, Kfir Levy +1
In Reinforcement Learning (RL), the goal of agents is to discover an optimal policy that maximizes the expected cumulative rewards. This objective may also be viewed as finding a p…
Efficient Policy Iteration for Robust Markov Decision Processes via Regularization
Navdeep Kumar, Kfir Levy, Kaixin Wang +1
Robust Markov decision processes (MDPs) provide a general framework to model decision problems where the system dynamics are changing or only partially known. Efficient methods for…
The Geometry of Robust Value Functions
Kaixin Wang, Navdeep Kumar, Kuangqi Zhou +3
The space of value functions is a fundamental concept in reinforcement learning. Characterizing its geometric properties may provide insights for optimization and representation. E…