3 citations · 10 across the 17 of their papers we have counts for
19 papers
V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Control
Donghu Kim, Youngdo Lee, Hojoon Lee +6
Improving sample efficiency remains a core challenge in reinforcement learning (RL), especially in real-world settings like robotics, where data collection is costly. This challeng…
Local Guidance, Global Impact: Gaussian-Reshaped Trust Region Unlocks Behavior Transitions
Bingxu Liu, Jiashun Liu, Johan Obando-Ceron +5
While Proximal Policy Optimization (PPO) demonstrates strong performance in stationary settings, we show that its standard optimization paradigm struggles in continual and non-stat…
A Mechanistic Analysis of Looped Reasoning Language Models
Hugh Blayney, Álvaro Arroyo, Johan Obando-Ceron +4
Reasoning has become a central capability in large language models. Recent research has shown that reasoning performance can be improved by looping an LLM's layers in the latent di…
Stable Deep Reinforcement Learning via Isotropic Gaussian Representations
Ali Saheb Pasand, Johan Obando-Ceron, Aaron Courville +2
Deep reinforcement learning systems often suffer from unstable training dynamics due to non-stationarity, where learning objectives and data distributions evolve over time. We show…
Asymmetric Proximal Policy Optimization: mini-critics boost LLM reasoning
Jiashun Liu, Johan Obando-Ceron, Han Lu +7
Most recent RL for LLMs (RL4LLM) methods avoid explicit critics, replacing them with average advantage baselines. This shift is largely pragmatic: conventional value functions are…
Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agents
Johan Obando-Ceron, Walter Mayor, Samuel Lavoie +3
Recent works have proposed accelerating the wall-clock training time of actor-critic methods via the use of large-scale environment parallelization; unfortunately, these can someti…
The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learning
Jiashun Liu, Johan Obando-Ceron, Pablo Samuel Castro +2
Off-policy deep reinforcement learning (RL) typically leverages replay buffers for reusing past experiences during learning. This can help improve sample efficiency when the collec…
The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networks
Walter Mayor, Johan Obando-Ceron, Aaron Courville +1
The use of parallel actors for data collection has been an effective technique used in reinforcement learning (RL) algorithms. The manner in which data is collected in these algori…
Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning
Roger Creus Castanyer, Johan Obando-Ceron, Lu Li +4
Scaling deep reinforcement learning networks is challenging and often results in degraded performance, yet the root causes of this failure mode remain poorly understood. Several re…
Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churn
Hongyao Tang, Johan Obando-Ceron, Pablo Samuel Castro +2
Plasticity, or the ability of an agent to adapt to new tasks, environments, or distributions, is crucial for continual learning. In this paper, we study the loss of plasticity in d…
Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning
Jiashun Liu, Zihao Wu, Johan Obando-Ceron +3
Deep reinforcement learning (RL) agents frequently suffer from neuronal activity loss, which impairs their ability to adapt to new data and learn continually. A common method to qu…
CALE: Continuous Arcade Learning Environment
Jesse Farebrother, Pablo Samuel Castro
We introduce the Continuous Arcade Learning Environment (CALE), an extension of the well-known Arcade Learning Environment (ALE) [Bellemare et al., 2013]. The CALE uses the same un…
Mixture of Experts in a Mixture of RL settings
Timon Willi, Johan Obando-Ceron, Jakob Foerster +2
Mixtures of Experts (MoEs) have gained prominence in (self-)supervised learning due to their enhanced inference efficiency, adaptability to distributed training, and modularity. Pr…
On the consistency of hyper-parameter selection in value-based deep reinforcement learning
Johan Obando-Ceron, João G. M. Araújo, Aaron Courville +1
Deep reinforcement learning (deep RL) has achieved tremendous success on various domains through a combination of algorithmic design and careful selection of hyper-parameters. Algo…
Stop Regressing: Training Value Functions via Classification for Scalable Deep RL
Jesse Farebrother, Jordi Orbay, Quan Vuong +9
Value functions are a central component of deep reinforcement learning (RL). These functions, parameterized by neural networks, are trained using a mean squared error regression ob…
In value-based deep reinforcement learning, a pruned network is a good network
Johan Obando-Ceron, Aaron Courville, Pablo Samuel Castro
Recent work has shown that deep reinforcement learning agents have difficulty in effectively using their network parameters. We leverage prior insights into the advantages of spars…
Mixtures of Experts Unlock Parameter Scaling for Deep RL
Johan Obando-Ceron, Ghada Sokar, Timon Willi +6
The recent rapid progress in (self) supervised learning models is in large part predicted by empirical scaling laws: a model's performance scales proportionally to its size. Analog…
Learning and Controlling Silicon Dopant Transitions in Graphene using Scanning Transmission Electron Microscopy
Max Schwarzer, Jesse Farebrother, Joshua Greaves +8
We introduce a machine learning approach to determine the transition dynamics of silicon atoms on a single layer of carbon atoms, when stimulated by the electron beam of a scanning…
Small batch deep reinforcement learning
Johan Obando-Ceron, Marc G. Bellemare, Pablo Samuel Castro
In value-based deep reinforcement learning with replay memories, the batch size parameter specifies how many transitions to sample for each gradient update. Although critical to th…