984 citations · 3.3k across the 41 of their papers we have counts for
8 papers · 1 filter
Transformers generalize differently from information stored in context vs in weights
Stephanie C. Y. Chan, Ishita Dasgupta, Junkyung Kim +3
Transformer models can use two fundamentally different kinds of information: information stored in weights during training, and information provided ``in-context'' at inference tim…
Meaning without reference in large language models
Steven T. Piantadosi, Felix Hill
The widespread success of large language models (LLMs) has been met with skepticism that they possess anything like human concepts or meanings. Contrary to claims that LLMs possess…
Language models show human-like content effects on reasoning tasks
Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan +5
Reasoning is a key ability for an intelligent system. Large language models (LMs) achieve above-chance performance on abstract reasoning tasks, but exhibit many imperfections. Howe…
Know your audience: specializing grounded language models with listener subtraction
Aaditya K. Singh, David Ding, Andrew Saxe +2
Effective communication requires adapting to the idiosyncrasies of each communicative context--such as the common ground shared with each partner. Humans demonstrate this ability t…
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Stephanie C. Y. Chan, Adam Santoro, Andrew K. Lampinen +5
Large transformer-based models are able to perform in-context few-shot learning, without being explicitly trained for it. This observation raises the question: what aspects of the…
Semantic Exploration from Language Abstractions and Pretrained Representations
Allison C. Tam, Neil C. Rabinowitz, Andrew K. Lampinen +6
Effective exploration is a challenge in reinforcement learning (RL). Novelty-based exploration methods can suffer in high-dimensional state spaces, such as continuous partially-obs…
Can language models learn from explanations in context?
Andrew K. Lampinen, Ishita Dasgupta, Stephanie C. Y. Chan +6
Language Models (LMs) can perform new tasks by adapting to a few in-context examples. For humans, explanations that connect examples to task principles can improve learning. We the…
Zipfian environments for Reinforcement Learning
Stephanie C. Y. Chan, Andrew K. Lampinen, Pierre H. Richemond +1
As humans and animals learn in the natural world, they encounter distributions of entities, situations and events that are far from uniform. Typically, a relatively small set of ex…