7 papers · 1 filter
Learning to Control Coupled-Dynamics Environments with Joint Markov Decision Processes
Ege C. Kaya, Aliasghar Pourghani, Mahsa Ghasemi +2
Coupled-dynamics environments expose the one-step outcomes that would follow from several possible counterfactual actions under a common realization of exogenous randomness. The or…
Quotient-Categorical Representations for Bellman-Compatible Average-Reward Distributional Reinforcement Learning
Ege C. Kaya, Aliasghar Pourghani, Vijay Gupta +1
Average-reward reinforcement learning requires estimating the gain and the bias, which is defined only up to an additive constant. This makes direct distributional analogues ill-po…
A Finite-Iteration Theory for Asynchronous Categorical Distributional Temporal-Difference Learning
Ege C. Kaya, Abolfazl Hashemi
We study finite-iteration behavior of the exact asynchronous recursions used by categorical distributional temporal-difference methods. The analysis covers scalar categorical TD in…
Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance
Arda Fazla, Ege C. Kaya, Antesh Upadhyay +1
Analysis of Stochastic Gradient Descent (SGD) and its variants typically relies on the assumption of uniformly bounded variance, a condition that frequently fails in practical non-…
Joint MDPs and Reinforcement Learning in Coupled-Dynamics Environments
Ege C. Kaya, Mahsa Ghasemi, Abolfazl Hashemi
Many distributional quantities in reinforcement learning are intrinsically joint across actions, including distributions of gaps and probabilities of superiority. However, the clas…
Localized Distributional Robustness in Submodular Multi-Task Subset Selection
Ege C. Kaya, Abolfazl Hashemi
In this work, we treat the problem of multi-task submodular optimization from the perspective of local distributional robustness within the neighborhood of a reference distribution…