On the Limited Representational Power of Value Functions and its Links to Statistical (In)Efficiency

David Cheikhi, Daniel Russo

Identifying the trade-offs between model-based and model-free methods is a central question in reinforcement learning. Value-based methods offer substantial computational advantage…