Shared Actors Need Not Share Critics: Effects of Value Mismatch in Parallel Reinforcement Learning

Zhenya Liu, Yang Meng, Zhuokai Zhao +2

When a single policy is trained in parallel across multiple environments of the same task, such as procedurally generated levels, randomized dynamics, or curricula, implementations…