54 citations · 221 across the 11 of their papers we have counts for
11 papers
The Transient Nature of Emergent In-Context Learning in Transformers
Aaditya K. Singh, Stephanie C. Y. Chan, Ted Moskovitz +3
Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has provided a deeper understand…
Passive learning of active causal strategies in agents and language models
Andrew Kyle Lampinen, Stephanie C Y Chan, Ishita Dasgupta +2
What can be learned about causality and experimentation from passive data? This question is salient given recent successes of passively-trained language models in interactive domai…
Transformers generalize differently from information stored in context vs in weights
Stephanie C. Y. Chan, Ishita Dasgupta, Junkyung Kim +3
Transformer models can use two fundamentally different kinds of information: information stored in weights during training, and information provided ``in-context'' at inference tim…
Language models show human-like content effects on reasoning tasks
Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan +5
Reasoning is a key ability for an intelligent system. Large language models (LMs) achieve above-chance performance on abstract reasoning tasks, but exhibit many imperfections. Howe…
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Stephanie C. Y. Chan, Adam Santoro, Andrew K. Lampinen +5
Large transformer-based models are able to perform in-context few-shot learning, without being explicitly trained for it. This observation raises the question: what aspects of the…
Semantic Exploration from Language Abstractions and Pretrained Representations
Allison C. Tam, Neil C. Rabinowitz, Andrew K. Lampinen +6
Effective exploration is a challenge in reinforcement learning (RL). Novelty-based exploration methods can suffer in high-dimensional state spaces, such as continuous partially-obs…
Can language models learn from explanations in context?
Andrew K. Lampinen, Ishita Dasgupta, Stephanie C. Y. Chan +6
Language Models (LMs) can perform new tasks by adapting to a few in-context examples. For humans, explanations that connect examples to task principles can improve learning. We the…
Zipfian environments for Reinforcement Learning
Stephanie C. Y. Chan, Andrew K. Lampinen, Pierre H. Richemond +1
As humans and animals learn in the natural world, they encounter distributions of entities, situations and events that are far from uniform. Typically, a relatively small set of ex…
Tell me why! Explanations support learning relational and causal structure
Andrew K. Lampinen, Nicholas A. Roy, Ishita Dasgupta +8
Inferring the abstract relational and causal structure of the world is a major challenge for reinforcement-learning (RL) agents. For humans, language--particularly in the form of e…
Towards mental time travel: a hierarchical memory for reinforcement learning agents
Andrew Kyle Lampinen, Stephanie C. Y. Chan, Andrea Banino +1
Reinforcement learning agents often forget details of the past, especially after delays or distractor tasks. Agents with common memory architectures struggle to recall and integrat…
Measuring the Reliability of Reinforcement Learning Algorithms
Stephanie C. Y. Chan, Samuel Fishman, John Canny +2
Lack of reliability is a well-known issue for reinforcement learning (RL) algorithms. This problem has gained increasing attention in recent years, and efforts to improve it have g…