5 citations · 5 across the 2 of their papers we have counts for
4 papers
Stochastic Approximation with Markov Noise: Analysis and applications in reinforcement learning
Prasenjit Karmakar
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slowe…
Customer-server population dynamics in heavy traffic
Rami Atar, Prasenjit Karmakar, David Lipshutz
We study a many-server queueing model with server vacations, where the population size dynamics of servers and customers are coupled: a server may leave for vacation only when no c…
On a convergent off -policy temporal difference learning algorithm in on-line learning environment
Prasenjit Karmakar, Rajkumar Maity, Shalabh Bhatnagar
In this paper we provide a rigorous convergence analysis of a "off"-policy temporal difference learning algorithm with linear function approximation and per time-step linear comput…
Are Markov Models Effective for Storage Reliability Modelling?
Prasenjit Karmakar, K. Gopinath
Continuous Time Markov Chains (CTMC) have been used extensively to model reliability of storage systems. While the exponentially distributed sojourn time of Markov models is widely…