28 citations · 31 across the 12 of their papers we have counts for
8 papers · 1 filter
Bridging the Gap Between Average and Discounted TD Learning
Haoxing Tian, Zaiwei Chen, Ioannis Ch. Paschalidis +1
The analysis of Temporal Difference (TD) learning in the average-reward setting faces notable theoretical difficulties because the Bellman operator is not contractive with respect…
Geometric Re-Analysis of Classical MDP Solving Algorithms
Arsenii Mustafin, Aleksei Pakharev, Alex Olshevsky +1
We build on a recently introduced geometric interpretation of Markov Decision Processes (MDPs) to analyze classical MDP-solving algorithms: Value Iteration (VI) and Policy Iteratio…
Analysis of Value Iteration Through Absolute Probability Sequences
Arsenii Mustafin, Sebastien Colla, Alex Olshevsky +1
Value Iteration is a widely used algorithm for solving Markov Decision Processes (MDPs). While previous studies have extensively analyzed its convergence properties, they primarily…
MDP Geometry, Normalization and Reward Balancing Solvers
Arsenii Mustafin, Aleksei Pakharev, Alex Olshevsky +1
We present a new geometric interpretation of Markov Decision Processes (MDPs) with a natural normalization procedure that allows us to adjust the value function at each state witho…
On Value Iteration Convergence in Connected MDPs
Arsenii Mustafin, Alex Olshevsky, Ioannis Ch. Paschalidis
This paper establishes that an MDP with a unique optimal policy and ergodic associated transition matrix ensures the convergence of various versions of the Value Iteration algorith…
One-Shot Averaging for Distributed TD() Under Markov Sampling
Haoxing Tian, Ioannis Ch. Paschalidis, Alex Olshevsky
We consider a distributed setup for reinforcement learning, where each agent has a copy of the same Markov Decision Process but transitions are sampled from the corresponding Marko…
On the Performance of Temporal Difference Learning With Neural Networks
Haoxing Tian, Ioannis Ch. Paschalidis, Alex Olshevsky
Neural Temporal Difference (TD) Learning is an approximate temporal difference method for policy evaluation that uses a neural network for function approximation. Analysis of Neura…
Closing the gap between SVRG and TD-SVRG with Gradient Splitting
Arsenii Mustafin, Alex Olshevsky, Ioannis Ch. Paschalidis
Temporal difference (TD) learning is a policy evaluation in reinforcement learning whose performance can be enhanced by variance reduction methods. Recently, multiple works have so…