Adaptive Estimation and Optimal Control in Offline Contextual MDPs without Stationarity

Riddhiman Bhattacharyya, Sayak Chakrabarty, Imon Banerjee

Contextual MDPs are powerful tools with wide applicability in areas from biostatistics to machine learning. However, specializing them to offline datasets has been challenging due…