Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning

Lars van der Laan, Nathan Kallus

Marginalized importance weighting evaluates a target policy by reweighting offline state-action samples with its discounted occupancy ratio, characterized by an adjoint Bellman equ…