978 citations · 1.4k across the 15 of their papers we have counts for
15 papers
Causal interventions expose implicit situation models for commonsense language understanding
Takateru Yamakoshi, James L. McClelland, Adele E. Goldberg +1
Accounts of human language processing have long appealed to implicit ``situation models'' that enrich comprehension with relevant but unstated world knowledge. Here, we apply causa…
Achieving and Understanding Out-of-Distribution Generalization in Systematic Reasoning in Small-Scale Transformers
Andrew J. Nam, Mustafa Abdool, Trevor Maxfield +1
Out-of-distribution generalization (OODG) is a longstanding challenge for neural networks. This challenge is quite apparent in tasks with well-defined variables and rules, where ex…
Learning to Reason With Relational Abstractions
Andrew J. Nam, Mengye Ren, Chelsea Finn +1
Large language models have recently shown promising progress in mathematical reasoning when fine-tuned with human-generated sequences walking through a sequence of solution steps.…
Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks
Yuxuan Li, James L. McClelland
Transformer networks have seen great success in natural language processing and machine vision, where task objectives such as next word prediction and image classification benefit…
Language models show human-like content effects on reasoning tasks
Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan +5
Reasoning is a key ability for an intelligent system. Large language models (LMs) achieve above-chance performance on abstract reasoning tasks, but exhibit many imperfections. Howe…
Can language models learn from explanations in context?
Andrew K. Lampinen, Ishita Dasgupta, Stephanie C. Y. Chan +6
Language Models (LMs) can perform new tasks by adapting to a few in-context examples. For humans, explanations that connect examples to task principles can improve learning. We the…
Tell me why! Explanations support learning relational and causal structure
Andrew K. Lampinen, Nicholas A. Roy, Ishita Dasgupta +8
Inferring the abstract relational and causal structure of the world is a major challenge for reinforcement-learning (RL) agents. For humans, language--particularly in the form of e…
Systematic human learning and generalization from a brief tutorial with explanatory feedback
Andrew J. Nam, James L. McClelland
Neural networks have long been used to model human intelligence, capturing elements of behavior and cognition, and their neural basis. Recent advancements in deep learning have ena…
Transforming task representations to perform novel tasks
Andrew K. Lampinen, James L. McClelland
An important aspect of intelligence is the ability to adapt to a novel task without any direct experience (zero-shot), based on its relationship to previous tasks. Humans can exhib…
Extending Machine Language Models toward Human-Level Language Understanding
James L. McClelland, Felix Hill, Maja Rudolph +2
Language is crucial for human intelligence, but what exactly is its role? We take language to be a part of a system for understanding and communicating about situations. The human…
Environmental drivers of systematicity and generalization in a situated agent
Felix Hill, Andrew Lampinen, Rosalia Schneider +4
The question of whether deep neural networks are good at generalising beyond their immediate training experience is of critical importance for learning-based approaches to AI. Here…
Zero-shot task adaptation by homoiconic meta-mapping
Andrew K. Lampinen, James L. McClelland
How can deep learning systems flexibly reuse their knowledge? Toward this goal, we propose a new class of challenges, and a class of architectures that can solve them. The challeng…
A mathematical theory of semantic development in deep neural networks
Andrew M. Saxe, James L. McClelland, Surya Ganguli
An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, th…
One-shot and few-shot learning of word embeddings
Andrew K. Lampinen, James L. McClelland
Standard deep learning systems require thousands or millions of examples to learn a concept, and cannot integrate new concepts easily. By contrast, humans have an incredible abilit…
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Andrew M. Saxe, James L. McClelland, Surya Ganguli
Despite the widespread practical success of deep learning methods, our theoretical understanding of the dynamics of learning in deep neural networks remains quite sparse. We attemp…