24 citations · 75 across the 22 of their papers we have counts for
15 papers · 1 filter
Learning to Solve Hard Problems in RL for LLMs by Never Giving Up
Michael Noukhovitch, Hamish Ivison, Nathan Lambert +1
We demonstrate that training LLMs with RL does not improve performance equally across a dataset. RL shows large improvements on easy problems that an LLM is already good at solving…
Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
Michael Noukhovitch, Shengyi Huang, Sophie Xhonneux +3
The dominant paradigm for RLHF is online and on-policy RL: synchronously generating from the large language model (LLM) policy, labelling with a reward model, and learning using fe…
Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RL
Ghada Sokar, Johan Obando-Ceron, Aaron Courville +2
The use of deep neural networks in reinforcement learning (RL) often suffers from performance degradation as model size increases. While soft mixtures of experts (SoftMoEs) have re…
On the consistency of hyper-parameter selection in value-based deep reinforcement learning
Johan Obando-Ceron, João G. M. Araújo, Aaron Courville +1
Deep reinforcement learning (deep RL) has achieved tremendous success on various domains through a combination of algorithmic design and careful selection of hyper-parameters. Algo…
In value-based deep reinforcement learning, a pruned network is a good network
Johan Obando-Ceron, Aaron Courville, Pablo Samuel Castro
Recent work has shown that deep reinforcement learning agents have difficulty in effectively using their network parameters. We leverage prior insights into the advantages of spars…
V-STaR: Training Verifiers for Self-Taught Reasoners
Arian Hosseini, Xingdi Yuan, Nikolay Malkin +3
Common self-improvement approaches for large language models (LLMs), such as STaR, iteratively fine-tune LLMs on self-generated solutions to improve their problem-solving ability.…
Meta-Value Learning: a General Framework for Learning with Learning Awareness
Tim Cooijmans, Milad Aghajohari, Aaron Courville
Gradient-based learning in multi-agent systems is difficult because the gradient derives from a first-order model which does not account for the interaction between agents' learnin…
Bigger, Better, Faster: Human-level Atari with human-level efficiency
Max Schwarzer, Johan Obando-Ceron, Aaron Courville +3
We introduce a value-based RL agent, which we call BBF, that achieves super-human performance in the Atari 100K benchmark. BBF relies on scaling the neural networks used for value…
Teaching Algorithmic Reasoning via In-context Learning
Hattie Zhou, Azade Nova, Hugo Larochelle +3
Large language models (LLMs) have shown increasing in-context learning capabilities through scaling up model and data size. Despite this progress, LLMs are still unable to solve al…
Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress
Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro +2
Learning tabula rasa, that is without any prior knowledge, is the prevalent workflow in reinforcement learning (RL) research. However, RL systems, when applied to large-scale setti…
Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods
Yuchen Lu, Zhen Liu, Aristide Baratin +3
We address the problem of evaluating the quality of self-supervised learning (SSL) models without access to supervised labels, while being agnostic to the architecture, learning al…
The Primacy Bias in Deep Reinforcement Learning
Evgenii Nikishin, Max Schwarzer, Pierluca D'Oro +2
This work identifies a common flaw of deep reinforcement learning (RL) algorithms: a tendency to rely on early interactions and ignore useful evidence encountered later. Because of…
Simplicial Embeddings in Self-Supervised Learning and Downstream Classification
Samuel Lavoie, Christos Tsirigotis, Max Schwarzer +4
Simplicial Embeddings (SEM) are representations learned through self-supervised learning (SSL), wherein a representation is projected into simplices of dimensions each usin…
Iterated learning for emergent systematicity in VQA
Ankit Vani, Max Schwarzer, Yuchen Lu +2
Although neural module networks have an architectural bias towards compositionality, they require gold standard layouts to generalize systematically in practice. When instead learn…
Touch-based Curiosity for Sparse-Reward Tasks
Sai Rajeswar, Cyril Ibrahim, Nitin Surya +4
Robots in many real-world settings have access to force/torque sensors in their gripper and tactile sensing is often necessary in tasks that involve contact-rich motion. In this wo…