53 citations · 78 across the 4 of their papers we have counts for
9 papers
Transformers generalize differently from information stored in context vs in weights
Stephanie C. Y. Chan, Ishita Dasgupta, Junkyung Kim +3
Transformer models can use two fundamentally different kinds of information: information stored in weights during training, and information provided ``in-context'' at inference tim…
What shapes feature representations? Exploring datasets, architectures, and training
Katherine L. Hermann, Andrew K. Lampinen
In naturalistic learning problems, a model's input contains a wide range of features, some useful for the task at hand, and others not. Of the useful features, which ones does the…
Transforming task representations to perform novel tasks
Andrew K. Lampinen, James L. McClelland
An important aspect of intelligence is the ability to adapt to a novel task without any direct experience (zero-shot), based on its relationship to previous tasks. Humans can exhib…
Environmental drivers of systematicity and generalization in a situated agent
Felix Hill, Andrew Lampinen, Rosalia Schneider +4
The question of whether deep neural networks are good at generalising beyond their immediate training experience is of critical importance for learning-based approaches to AI. Here…
Automated curricula through setter-solver interactions
Sebastien Racaniere, Andrew K. Lampinen, Adam Santoro +3
Reinforcement learning algorithms use correlations between policies and rewards to improve agent performance. But in dynamic or sparsely rewarding environments these correlations a…
Zero-shot task adaptation by homoiconic meta-mapping
Andrew K. Lampinen, James L. McClelland
How can deep learning systems flexibly reuse their knowledge? Toward this goal, we propose a new class of challenges, and a class of architectures that can solve them. The challeng…