7.8k citations · 12.8k across the 52 of their papers we have counts for
20 papers · 1 filter
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
Aleksandar Botev, Soham De, Samuel L Smith +59
We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture. Griffin combines linear recurrences with local attention to achieve ex…
Hindering Adversarial Attacks with Implicit Neural Representations
Andrei A. Rusu, Dan A. Calian, Sven Gowal +1
We introduce the Lossy Implicit Network Activation Coding (LINAC) defence, an input transformation which successfully hinders several common adversarial attacks on CIFAR- class…
Probing Transfer in Deep Reinforcement Learning without Task Engineering
Andrei A. Rusu, Sebastian Flennerhag, Dushyant Rao +2
We evaluate the use of original game curricula supported by the Atari 2600 console as a heterogeneous transfer benchmark for deep reinforcement learning agents. Game designers crea…
MO2: Model-Based Offline Options
Sasha Salter, Markus Wulfmeier, Dhruva Tirumala +4
The ability to discover useful behaviours from past experience and transfer them to new tasks is considered a core component of natural embodied intelligence. Inspired by neuroscie…
The CLRS Algorithmic Reasoning Benchmark
Petar Veličković, Adrià Puigdomènech Badia, David Budden +5
Learning representations of algorithms is an emerging area of machine learning, seeking to bridge concepts from neural networks with classical algorithms. Several important works h…
Learning Transferable Motor Skills with Hierarchical Latent Mixture Policies
Dushyant Rao, Fereshteh Sadeghi, Leonard Hasenclever +8
For robots operating in the real world, it is desirable to learn reusable behaviours that can effectively be transferred and adapted to numerous tasks and scenarios. We propose an…
Reasoning-Modulated Representations
Petar Veličković, Matko Bošnjak, Thomas Kipf +4
Neural networks leverage robust internal representations in order to generalise. Learning them is difficult, and often requires a large training set that covers the data distributi…
On Multi-objective Policy Optimization as a Tool for Reinforcement Learning: Case Studies in Offline RL and Finetuning
Abbas Abdolmaleki, Sandy H. Huang, Giulia Vezzani +11
Many advances that have improved the robustness and efficiency of deep reinforcement learning (RL) algorithms can, in one way or another, be understood as introducing additional ob…
Skillful Precipitation Nowcasting using Deep Generative Models of Radar
Suman Ravuri, Karel Lenc, Matthew Willson +17
Precipitation nowcasting, the high-resolution forecasting of precipitation up to two hours ahead, supports the real-world socio-economic needs of many sectors reliant on weather-de…
A Distributional View on Multi-Objective Policy Optimization
Abbas Abdolmaleki, Sandy H. Huang, Leonard Hasenclever +7
Many real-world problems require trading off multiple competing objectives. However, these objectives are often in different units and/or scales, which can make it challenging for…
Disentangled Cumulants Help Successor Representations Transfer to New Tasks
Christopher Grimm, Irina Higgins, Andre Barreto +5
Biological intelligence can learn to solve many diverse tasks in a data efficient manner by re-using basic knowledge and skills from one task to another. Furthermore, many of such…
Continual Unsupervised Representation Learning
Dushyant Rao, Francesco Visin, Andrei A. Rusu +3
Continual learning aims to improve the ability of modern learning systems to deal with non-stationary distributions, typically by attempting to learn a series of tasks sequentially…
Stabilizing Transformers for Reinforcement Learning
Emilio Parisotto, H. Francis Song, Jack W. Rae +10
Owing to their ability to both effectively integrate information over long time horizons and scale to massive amounts of data, self-attention architectures have recently shown brea…
Meta-Learning with Warped Gradient Descent
Sebastian Flennerhag, Andrei A. Rusu, Razvan Pascanu +3
Learning an efficient update rule from data that promotes rapid learning of new tasks from the same distribution remains an open problem in meta-learning. Typically, previous works…
Meta-Learning with Latent Embedding Optimization
Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski +4
Gradient-based meta-learning techniques are both widely applicable and proficient at solving challenging few-shot learning and fast adaptation problems. However, they have practica…
Graph networks as learnable physics engines for inference and control
Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg +4
Understanding and interacting with everyday physical scenes requires rich knowledge about the structure of the world, represented either implicitly in a value or policy function, o…
Distral: Robust Multitask Reinforcement Learning
Yee Whye Teh, Victor Bapst, Wojciech Marian Czarnecki +5
Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data…
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz +11
The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Neural networks are not, in general, capable of this and it has been wi…
Progressive Neural Networks
Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins +5
Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence. The…
Policy Distillation
Andrei A. Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre +6
Policies for complex visual tasks have been successfully learned with deep reinforcement learning, using an approach called deep Q-networks (DQN), but relatively large (task-specif…