6 citations · 15 across the 20 of their papers we have counts for
23 papers
Mean-Field Control with a Common Hidden State under Decentralized Observations
Erhan Bayraktar, Ali D. Kara
We study optimal control of a system with multiple decision makers who share a common hidden state and receive fully decentralized observations through identical channels. The dyna…
Reinforcement Learning for Discounted and Ergodic Control of Diffusion Processes
Erhan Bayraktar, Ali D. Kara, Somnath Pradhan +1
This paper develops a quantized Q-learning algorithm for the optimal control of controlled diffusion processes on under both discounted and ergodic (average) cost cr…
Reinforcement Learning with Function Approximation for Non-Markov Processes
Ali Devran Kara
We study reinforcement learning methods with linear function approximation under non-Markov state and cost processes. We first consider the policy evaluation method and show that t…
Quantizer Design for Finite Model Approximations, Model Learning, and Quantized Q-Learning for MDPs with Unbounded Spaces
Osman Bicer, Ali D. Kara, Serdar Yuksel
In this paper, for Markov decision processes (MDPs) with unbounded state spaces we present refined upper bounds presented in [Kara et. al. JMLR'23] on finite model approximation er…
Sensitivity of Filter Kernels and Robustness Bounds to Transition and Measurement Kernel Perturbations in Partially Observable Stochastic Control
Yunus Emre Demirci, Ali Devran Kara, Serdar Yüksel
Studying the stability of partially observed Markov decision processes (POMDPs) with respect to perturbations in either transition or observation kernels is a significant problem.…
Learning POMDPs with Linear Function Approximation and Finite Memory
Ali Devran Kara
We study reinforcement learning with linear function approximation and finite-memory approximations for partially observed Markov decision processes (POMDPs). We first present an a…
Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning
Ali Devran Kara, Serdar Yuksel
In this review/tutorial article, we present recent progress on optimal control of partially observed Markov Decision Processes (POMDPs). We first present regularity and continuity…
Near Optimal Approximations and Finite Memory Policies for POMPDs with Continuous Spaces
Ali Devran Kara, Erhan Bayraktar, Serdar Yuksel
We study an approximation method for partially observed Markov decision processes (POMDPs) with continuous spaces. Belief MDP reduction, which has been the standard approach to stu…
Refined Bounds on Near Optimality Finite Window Policies in POMDPs and Their Reinforcement Learning
Yunus Emre Demirci, Ali Devran Kara, Serdar Yüksel
Finding optimal policies for Partially Observable Markov Decision Processes (POMDPs) is challenging due to their uncountable state spaces when transformed into fully observable Mar…
Learning with Linear Function Approximations in Mean-Field Control
Erhan Bayraktar, Ali D. Kara
The paper focuses on mean-field type multi-agent control problems with finite state and action spaces where the dynamics and cost structures are symmetric and homogeneous, and are…
Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations
Yunus Emre Demirci, Ali Devran Kara, Serdar Yüksel
The average cost optimality is known to be a challenging problem for partially observable stochastic control, with few results available beyond the finite state, action, and measur…
Q-Learning for Stochastic Control under General Information Structures and Non-Markovian Environments
Ali Devran Kara, Serdar Yuksel
As a primary contribution, we present a convergence theorem for stochastic iterations, and in particular, Q-learning iterates, under a general, possibly non-Markovian, stochastic e…
Infinite Horizon Average Cost Optimality Criteria for Mean-Field Control
Erhan Bayraktar, Ali D. Kara
We study mean-field control problems in discrete-time under the infinite horizon average cost optimality criteria. We focus on both the finite population and the infinite populatio…
Approximations and Learning for Continuous State and Action MDPs under Average Cost Criteria
Ali Devran Kara, Serdar Yuksel
In this paper, for Markov Decision Processes (MDPs) with standard Borel spaces, (i) we first provide a discretization based approximation method for MDPs with continuous spaces und…
Finite Approximations for Mean Field Type Multi-Agent Control and Their Near Optimality
Erhan Bayraktar, Nicole Bauerle, Ali Devran Kara
We study a multi-agent mean field type control problem in discrete time where the agents aim to find a socially optimal strategy and where the state and action spaces for the agent…
Approximate Q-Learning for Controlled Diffusion Processes and its Near Optimality
Erhan Bayraktar, Ali Devran Kara
We study a Q learning algorithm for continuous time stochastic control problems. The proposed algorithm uses the sampled state process by discretizing the state and control action…
Q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuity
Ali Devran Kara, Naci Saldi, Serdar Yüksel
Reinforcement learning algorithms often require finiteness of state and action spaces in Markov decision processes (MDPs) (also called controlled Markov chains) and various efforts…
Convergence of Finite Memory Q-Learning for POMDPs and Near Optimality of Learned Policies under Filter Stability
Ali Devran Kara, Serdar Yuksel
In this paper, for POMDPs, we provide the convergence of a Q learning algorithm for control policies using a finite history of past observations and control actions, and, consequen…
Near Optimality of Finite Memory Feedback Policies in Partially Observed Markov Decision Processes
Ali Devran Kara, Serdar Yuksel
In the theory of Partially Observed Markov Decision Processes (POMDPs), existence of optimal policies have in general been established via converting the original partially observe…
Robustness to Incorrect Models and Data-Driven Learning in Average-Cost Optimal Stochastic Control
Ali Devran Kara, Maxim Raginsky, Serdar Yuksel
We study continuity and robustness properties of infinite-horizon average expected cost problems with respect to (controlled) transition kernels, and applications of these results…
Weak Feller Property of Non-linear Filters
Ali Devran Kara, Naci Saldi, Serdar Yüksel
Weak Feller property of controlled and control-free Markov chains lead to many desirable properties. In control-free setups this leads to the existence of invariant probability mea…
Robustness to incorrect system models in stochastic control
Ali Devran Kara, Serdar Yüksel
In stochastic control applications, typically only an ideal model (controlled transition kernel) is assumed and the control design is based on the given model, raising the problem…
Robustness to incorrect priors in partially observed stochastic control
Ali Devran Kara, Serdar Yüksel
We study the continuity properties of optimal solutions to stochastic control problems with respect to initial probability measures and applications of these to the robustness of o…