5 papers
Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environments
Michael Beukman, Khimya Khetarpal, Zeyu Zheng +4
An agent's performance stagnating at a suboptimal level is a common problem in deep on-policy RL. Focusing on PPO, we show that plateaus in certain regimes arise not because of kno…
Beyond Fixed Tasks: Unsupervised Environment Design for Task-Level Pairs
Daniel Furelos-Blanco, Charles Pert, Frederik Kelbel +3
Training general agents to follow complex instructions (tasks) in intricate environments (levels) remains a core challenge in reinforcement learning. Random sampling of task-level…
Mitigating Goal Misgeneralization via Minimax Regret
Karim Abdel Sadek, Matthew Farrugia-Roberts, Usman Anwar +4
Safe generalization in reinforcement learning requires not only that a learned policy acts capably in new situations, but also that it uses its capabilities towards the pursuit of…
Refining Minimax Regret for Unsupervised Environment Design
Michael Beukman, Samuel Coward, Michael Matthews +4
In unsupervised environment design, reinforcement learning agents are trained on environment configurations (levels) generated by an adversary that maximises some objective. Regret…
minimax: Efficient Baselines for Autocurricula in JAX
Minqi Jiang, Michael Dennis, Edward Grefenstette +1
Unsupervised environment design (UED) is a form of automatic curriculum learning for training robust decision-making agents to zero-shot transfer into unseen environments. Such aut…