9 citations · 13 across the 3 of their papers we have counts for
3 papers
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
David Raposo, Sam Ritter, Blake Richards +3
Transformer-based language models spread FLOPs uniformly across input sequences. In this work we demonstrate that transformers can instead learn to dynamically allocate FLOPs (or c…
Tell me why! Explanations support learning relational and causal structure
Andrew K. Lampinen, Nicholas A. Roy, Ishita Dasgupta +8
Inferring the abstract relational and causal structure of the world is a major challenge for reinforcement-learning (RL) agents. For humans, language--particularly in the form of e…
Is coding a relevant metaphor for building AI? A commentary on "Is coding a relevant metaphor for the brain?", by Romain Brette
Adam Santoro, Felix Hill, David Barrett +3
Brette contends that the neural coding metaphor is an invalid basis for theories of what the brain does. Here, we argue that it is an insufficient guide for building an artificial…