61 citations · 73 across the 2 of their papers we have counts for
3 papers · 1 filter
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
David Raposo, Sam Ritter, Blake Richards +3
Transformer-based language models spread FLOPs uniformly across input sequences. In this work we demonstrate that transformers can instead learn to dynamically allocate FLOPs (or c…
Synthetic Returns for Long-Term Credit Assignment
David Raposo, Sam Ritter, Adam Santoro +5
Since the earliest days of reinforcement learning, the workhorse method for assigning credit to actions over time has been temporal-difference (TD) learning, which propagates credi…
Rapid Task-Solving in Novel Environments
Sam Ritter, Ryan Faulkner, Laurent Sartran +3
We propose the challenge of rapid task-solving in novel environments (RTS), wherein an agent must solve a series of tasks as rapidly as possible in an unfamiliar environment. An ef…