51 citations · 233 across the 19 of their papers we have counts for
19 papers
On Strategic Measures and Optimality Properties in Discrete-Time Stochastic Control with Universally Measurable Policies
Huizhen Yu
This paper concerns discrete-time infinite-horizon stochastic control systems with Borel state and action spaces and universally measurable policies. We study optimization problems…
Average-Cost Optimality Results for Borel-Space Markov Decision Processes with Universally Measurable Policies
Huizhen Yu
We consider discrete-time Markov Decision Processes with Borel state and action spaces and universally measurable policies. For several long-run average cost criteria, we establish…
Average Cost Optimality Inequality for Markov Decision Processes with Borel Spaces and Universally Measurable Policies
Huizhen Yu
We consider average-cost Markov decision processes (MDPs) with Borel state and action spaces and universally measurable policies. For the nonnegative cost model and an unbounded co…
On Linear Programming for Constrained and Unconstrained Average-Cost Markov Decision Processes with Countable Action Spaces and Strictly Unbounded Costs
Huizhen Yu
We consider the linear programming approach for constrained and unconstrained Markov decision processes (MDPs) under the long-run average cost criterion, where the class of MDPs in…
On the Minimum Pair Approach for Average-Cost Markov Decision Processes with Countable Discrete Action Spaces and Strictly Unbounded Costs
Huizhen Yu
We consider average-cost Markov decision processes (MDPs) with Borel state spaces, countable, discrete action spaces, and strictly unbounded one-stage costs. For the minimum pair a…
On Markov Decision Processes with Borel Spaces and an Average Cost Criterion
Huizhen Yu
We consider average-cost Markov decision processes (MDPs) with Borel state and action spaces and universally measurable policies. For the nonnegative cost model and an unbounded co…
Two geometric input transformation methods for fast online reinforcement learning with neural nets
Sina Ghiassian, Huizhen Yu, Banafsheh Rafiee +1
We apply neural nets with ReLU gates in online reinforcement learning. Our goal is to train these networks in an incremental manner, without the computationally expensive experienc…
On Convergence of some Gradient-based Temporal-Differences Algorithms for Off-Policy Learning
Huizhen Yu
We consider off-policy temporal-difference (TD) learning methods for policy evaluation in Markov decision processes with finite spaces and discounted reward criteria, and we presen…
On Generalized Bellman Equations and Temporal-Difference Learning
Huizhen Yu, A. Rupam Mahmood, Richard S. Sutton
We consider off-policy temporal-difference (TD) learning in discounted Markov decision processes, where the goal is to evaluate a policy in a model-free way by using observations o…
Multi-step Off-policy Learning Without Importance Sampling Ratios
Ashique Rupam Mahmood, Huizhen Yu, Richard S. Sutton
To estimate the value functions of policies from exploratory data, most model-free off-policy algorithms rely on importance sampling, where the use of importance sampling ratios of…
Some Simulation Results for Emphatic Temporal-Difference Learning Algorithms
Huizhen Yu
This is a companion note to our recent study of the weak convergence properties of constrained emphatic temporal-difference learning (ETD) algorithms from a theoretic perspective.…
Weak Convergence Properties of Constrained Emphatic Temporal-difference Learning with Constant and Slowly Diminishing Stepsize
Huizhen Yu
We consider the emphatic temporal-difference (TD) algorithm, ETD(), for learning the value functions of stationary policies in a discounted, finite state and action Markov decis…
Emphatic Temporal-Difference Learning
A. Rupam Mahmood, Huizhen Yu, Martha White +1
Emphatic algorithms are temporal-difference learning algorithms that change their effective state distribution by selectively emphasizing and de-emphasizing their updates on differ…
On Convergence of Emphatic Temporal-Difference Learning
Huizhen Yu
We consider emphatic temporal-difference learning algorithms for policy evaluation in discounted Markov decision processes with finite spaces. Such algorithms were recently propose…
Stochastic Shortest Path Games and Q-Learning
Huizhen Yu
We consider a class of two-player zero-sum stochastic games with finite state and compact control spaces, which we call stochastic shortest path (SSP) games. They are undiscounted…
On Convergence of Value Iteration for a Class of Total Cost Markov Decision Processes
Huizhen Yu
We consider a general class of total cost Markov decision processes (MDP) in which the one-stage costs can have arbitrary signs, but the sum of the negative parts of the one-stage…
A Mixed Value and Policy Iteration Method for Stochastic Control with Universally Measurable Policies
Huizhen Yu, Dimitri P. Bertsekas
We consider stochastic control models with Borel spaces and universally measurable policies. For such models the standard policy iteration is known to have difficult measurability…
Discretized Approximations for POMDP with Average Cost
Huizhen Yu, Dimitri Bertsekas
In this paper, we propose a new lower approximation scheme for POMDP with discounted and average cost criterion. The approximating functions are determined by their values at a fin…
A Function Approximation Approach to Estimation of Policy Gradient for POMDP with Structured Policies
Huizhen Yu
We consider the estimation of the policy gradient in partially observable Markov decision processes (POMDP) with a special class of structured policies that are finite-state contro…