3.4k citations · 3.6k across the 22 of their papers we have counts for
23 papers
ShiQ: Bringing back Bellman to LLMs
Pierre Clavier, Nathan Grinsztajn, Raphael Avalos +8
The fine-tuning of pre-trained large language models (LLMs) using reinforcement learning (RL) is generally formulated as direct policy optimization. This approach was naturally fav…
Aya Vision: Advancing the Frontier of Multilingual Multimodality
Saurabh Dash, Yiyang Nan, John Dang +22
Building multimodal language models is fundamentally challenging: it requires aligning vision and language modalities, curating high-quality instruction data, and avoiding the degr…
Command A: An Enterprise-Ready Large Language Model
Team Cohere, :, Aakanksha +227
In this report we describe the development of Command A, a powerful large language model purpose-built to excel at real-world enterprise use cases. Command A is an agent-optimised…
Offline Regularised Reinforcement Learning for Large Language Models Alignment
Pierre Harvey Richemond, Yunhao Tang, Daniel Guo +15
The dominant framework for alignment of large language models (LLM), whether through reinforcement learning from human feedback or direct preference optimisation, is to learn from…
Human Alignment of Large Language Models through Online Preference Optimisation
Daniele Calandriello, Daniel Guo, Remi Munos +10
Ensuring alignment of language models' outputs with human preferences is critical to guarantee a useful, safe, and pleasant user experience. Thus, human alignment has been extensiv…
Generalized Preference Optimization: A Unified Approach to Offline Alignment
Yunhao Tang, Zhaohan Daniel Guo, Zeyu Zheng +7
Offline preference optimization allows fine-tuning large models directly from offline data, and has proved effective in recent alignment practices. We propose generalized preferenc…
The Edge of Orthogonality: A Simple View of What Makes BYOL Tick
Pierre H. Richemond, Allison Tam, Yunhao Tang +3
Self-predictive unsupervised learning methods such as BYOL or SimSiam have shown impressive results, and counter-intuitively, do not collapse to trivial representations. In this wo…
SemPPL: Predicting pseudo-labels for better contrastive representations
Matko Bošnjak, Pierre H. Richemond, Nenad Tomasev +7
Learning from large amounts of unsupervised data and a small amount of supervision is an important open problem in computer vision. We propose a new semi-supervised learning method…
Continuous diffusion for categorical data
Sander Dieleman, Laurent Sartran, Arman Roshannai +11
Diffusion models have quickly become the go-to paradigm for generative modelling of perceptual signals (such as images and sound) through iterative refinement. Their success hinges…
Understanding Self-Predictive Learning for Reinforcement Learning
Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond +13
We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their…
Categorical SDEs with Simplex Diffusion
Pierre H. Richemond, Sander Dieleman, Arnaud Doucet
Diffusion models typically operate in the standard framework of generative modelling by producing continuously-valued datapoints. To this end, they rely on a progressive Gaussian s…
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Stephanie C. Y. Chan, Adam Santoro, Andrew K. Lampinen +5
Large transformer-based models are able to perform in-context few-shot learning, without being explicitly trained for it. This observation raises the question: what aspects of the…
Zipfian environments for Reinforcement Learning
Stephanie C. Y. Chan, Andrew K. Lampinen, Pierre H. Richemond +1
As humans and animals learn in the natural world, they encounter distributions of entities, situations and events that are far from uniform. Typically, a relatively small set of ex…
BYOL works even without batch statistics
Pierre H. Richemond, Jean-Bastien Grill, Florent Altché +8
Bootstrap Your Own Latent (BYOL) is a self-supervised learning approach for image representation. From an augmented view of an image, BYOL trains an online network to predict a tar…
Bootstrap your own latent: A new approach to self-supervised Learning
Jean-Bastien Grill, Florian Strub, Florent Altché +11
We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target…
Biologically inspired architectures for sample-efficient deep reinforcement learning
Pierre H. Richemond, Arinbjörn Kolbeinsson, Yike Guo
Deep reinforcement learning requires a heavy price in terms of sample efficiency and overparameterization in the neural networks used for function approximation. In this work, we u…
Sample-Efficient Reinforcement Learning with Maximum Entropy Mellowmax Episodic Control
Marta Sarrico, Kai Arulkumaran, Andrea Agostinelli +2
Deep networks have enabled reinforcement learning to scale to more complex and challenging domains, but these methods typically require large quantities of training data. An altern…
Memory-Efficient Episodic Control Reinforcement Learning with Dynamic Online k-means
Andrea Agostinelli, Kai Arulkumaran, Marta Sarrico +2
Recently, neuro-inspired episodic control (EC) methods have been developed to overcome the data-inefficiency of standard deep reinforcement learning approaches. Using non-/semi-par…
Static Activation Function Normalization
Pierre H. Richemond, Yike Guo
Recent seminal work at the intersection of deep neural networks practice and random matrix theory has linked the convergence speed and robustness of these networks with the combina…
Combining learning rate decay and weight decay with complexity gradient descent - Part I
Pierre H. Richemond, Yike Guo
The role of regularization, in the specific case of deep neural networks rather than more traditional machine learning models, is still not fully elucidated. We hypothesize t…
A short variational proof of equivalence between policy gradients and soft Q learning
Pierre H. Richemond, Brendan Maginnis
Two main families of reinforcement learning algorithms, Q-learning and policy gradients, have recently been proven to be equivalent when using a softmax relaxation on one part, and…
On Wasserstein Reinforcement Learning and the Fokker-Planck equation
Pierre H. Richemond, Brendan Maginnis
Policy gradients methods often achieve better performance when the change in policy is limited to a small Kullback-Leibler divergence. We derive policy gradients where the change i…
Efficiently applying attention to sequential data with the Recurrent Discounted Attention unit
Brendan Maginnis, Pierre H. Richemond
Recurrent Neural Networks architectures excel at processing sequences by modelling dependencies over different timescales. The recently introduced Recurrent Weighted Average (RWA)…