61 citations · 110 across the 6 of their papers we have counts for
6 papers
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
David Raposo, Sam Ritter, Blake Richards +3
Transformer-based language models spread FLOPs uniformly across input sequences. In this work we demonstrate that transformers can instead learn to dynamically allocate FLOPs (or c…
How to Learn and Represent Abstractions: An Investigation using Symbolic Alchemy
Badr AlKhamissi, Akshay Srinivasan, Zeb-Kurth Nelson +1
Alchemy is a new meta-learning environment rich enough to contain interesting abstractions, yet simple enough to make fine-grained analysis tractable. Further, Alchemy provides an…
Synthetic Returns for Long-Term Credit Assignment
David Raposo, Sam Ritter, Adam Santoro +5
Since the earliest days of reinforcement learning, the workhorse method for assigning credit to actions over time has been temporal-difference (TD) learning, which propagates credi…
Rapid Task-Solving in Novel Environments
Sam Ritter, Ryan Faulkner, Laurent Sartran +3
We propose the challenge of rapid task-solving in novel environments (RTS), wherein an agent must solve a series of tasks as rapidly as possible in an unfamiliar environment. An ef…
Been There, Done That: Meta-Learning with Episodic Recall
Samuel Ritter, Jane X. Wang, Zeb Kurth-Nelson +4
Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks…
Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study
Samuel Ritter, David G. T. Barrett, Adam Santoro +1
Deep neural networks (DNNs) have achieved unprecedented performance on a wide range of complex tasks, rapidly outpacing our understanding of the nature of their solutions. This has…