4 papers
Model-based Bootstrap of Controlled Markov Chains
Ziwei Su, Imon Banerjee, Diego Klabjan
We propose and analyze a model-based bootstrap for transition kernels in finite controlled Markov chains (CMCs) with possibly nonstationary or history-dependent control policies, a…
Central Limit Theorems for Transition Probabilities of Controlled Markov Chains
Ziwei Su, Imon Banerjee, Diego Klabjan
We develop a central limit theorem (CLT) for a non-parametric estimator of the transition matrices in controlled Markov chains (CMCs) with finite state-action spaces. Our results e…
Offline Estimation of Controlled Markov Chains: Minimaxity and Sample Complexity
Imon Banerjee, Harsha Honnappa, Vinayak Rao
In this work, we study a natural nonparametric estimator of the transition probability matrices of a finite controlled Markov chain. We consider an offline setting with a fixed dat…
Adaptive Estimation of the Transition Density of Controlled Markov Chains
Imon Banerjee, Vinayak Rao, Harsha Honnappa
Estimating the transition dynamics of controlled Markov chains is crucial in fields such as time series analysis, reinforcement learning, and system exploration. Traditional non-pa…