18 citations · 41 across the 15 of their papers we have counts for
16 papers
Optimal Activation of Halting Multi-Armed Bandit Models
Wesley Cowan, Michael N. Katehakis, Sheldon M. Ross
We study new types of dynamic allocation problems the {\sl Halting Bandit} models. As an application, we obtain new proofs for the classic Gittins index decomposition result and re…
Accelerating the Computation of UCB and Related Indices for Reinforcement Learning
Wesley Cowan, Michael N. Katehakis, Daniel Pirutinsky
In this paper we derive an efficient method for computing the indices associated with an asymptotically optimal upper confidence bound algorithm (MDP-UCB) of Burnetas and Katehakis…
Reinforcement Learning: a Comparison of UCB Versus Alternative Adaptive Policies
Wesley Cowan, Michael N. Katehakis, Daniel Pirutinsky
In this paper we consider the basic version of Reinforcement Learning (RL) that involves computing optimal data driven (adaptive) policies for Markovian decision process with unkno…
Optimal Data Driven Resource Allocation under Multi-Armed Bandit Observations
Apostolos N. Burnetas, Odysseas Kanavetas, Michael N. Katehakis
This paper introduces the first asymptotically optimal strategy for a multi armed bandit (MAB) model under side constraints. The side constraints model situations in which bandit a…
Ameso Optimization: a Relaxation of Discrete Midpoint Convexity
Wen Chen, Odysseas Kanavetas, Michael N. Katehakis
In this paper we introduce the Ameso optimization problem, a special class of discrete optimization problems. We establish its basic properties and investigate the relation between…
Inventory Control Involving Unknown Demand of Discrete Nonperishable Items - Analysis of a Newsvendor-based Policy
Michael N. Katehakis, Jian Yang, Tingting Zhou
Inventory control with unknown demand distribution is considered, with emphasis placed on the case involving discrete nonperishable items. We focus on an adaptive policy which in e…
On the Solution to a Countable System of Equations Arising in Stochastic Processes
Michael N. Katehakis, Laurens C. Smit, Floske M. Spieksma
In this paper we develop a method to compute the solution to a countable (finite or infinite) set of equations that occurs in many different fields including Markov processes that…
Asymptotically Optimal Sequential Experimentation Under Generalized Ranking
Wesley Cowan, Michael N. Katehakis
We consider the \mnk{classical} problem of a controller activating (or sampling) sequentially from a finite number of populations, specified by unknown distributions. Ov…
Dynamic Pricing in a Dual Market Environment
Wen, Chen, Adam Fleischhacker +1
This paper is concerned with the determination of pricing strategies for a firm that in each period of a finite horizon receives replenishment quantities of a single product which…
Cash-Flow Based Dynamic Inventory Management
Michael N. Katehakis, Benjamin Melamed, Jim Shi
Small-to-medium size enterprises (SMEs), including many startup firms, need to manage interrelated flows of cash and inventories of goods. In this paper, we model a firm that can f…
Asymptotically Optimal Multi-Armed Bandit Policies under a Cost Constraint
Apostolos N. Burnetas, Odysseas Kanavetas, Michael N. Katehakis
We develop asymptotically optimal policies for the multi armed bandit (MAB), problem, under a cost constraint. This model is applicable in situations where each sample (or activati…
A Comparative Analysis of the Successive Lumping and the Lattice Path Counting Algorithms
Michael N. Katehakis, Laurens C. Smit, Floske M. Spieksma
This article provides a comparison of the successive lumping (SL) methodology with the popular lattice path counting algorithm in obtaining rate matrices for queueing models, satis…
Level product form QSF processes and an analysis of queues with Coxian inter-arrival distribution
Dwi Ertiningsih, Michael N. Katehakis, Laurens C. Smit +1
In this paper we study a class of Quasi-Skipfree (QSF) processes where the transition rate submatrices in the skipfree direction have a column times row structure. Under homogeneit…
Asymptotic Behavior of Minimal-Exploration Allocation Policies: Almost Sure, Arbitrarily Slow Growing Regret
Wesley Cowan, Michael N. Katehakis
The purpose of this paper is to provide further understanding into the structure of the sequential allocation ("stochastic multi-armed bandit", or MAB) problem by establishing prob…
An Asymptotically Optimal Policy for Uniform Bandits of Unknown Support
Wesley Cowan, Michael N. Katehakis
Consider the problem of a controller sampling sequentially from a finite number of populations, specified by random variables , and $k = 1, 2…
Normal Bandits of Unknown Means and Variances: Asymptotic Optimality, Finite Horizon Regret Bounds, and a Solution to an Open Problem
Wesley Cowan, Junya Honda, Michael N. Katehakis
Consider the problem of sampling sequentially from a finite number of populations, specified by random variables , and ; wh…