565 citations · 1.1k across the 47 of their papers we have counts for
49 papers
When Do Models Admit They Are Wrong? Failure Disclosure Is Unstable Under Reinforcement Learning
Steven Y. Feng, Noah D. Goodman, Michael C. Frank +3
Outcome-based reinforcement learning can produce models with similar task performance but very different ways of communicating about their mistakes. We study failure disclosure: wh…
Data augmentation as a framework for modeling hippocampal contributions to generalization
Tyler Bonnen, Andrew Kyle Lampinen
The hippocampus plays a critical role in generalization, enabling us to flexibly repurpose prior experiences to perform novel tasks. Here we suggest that data augmentation---a mach…
Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention
Jing Huang, Daniel Wurgaft, Rachit Bansal +6
Larger models learn tasks smaller models do not. What drives this phenomenon? We develop a simple phenomenological argument that power-law scaling already suggests that a larger mo…
Context Sensitivity Improves Human-Machine Visual Alignment
Frieda Born, Tom Neuhäuser, Lukas Muttenthaler +6
Modern machine learning models typically represent inputs as fixed points in a high-dimensional embedding space. While this approach has been proven powerful for a wide range of do…
Linear representations in language models can change dramatically over a conversation
Andrew Kyle Lampinen, Yuxuan Li, Eghbal Hosseini +2
Language model representations often contain linear directions that correspond to high-level concepts. Here, we study the dynamics of these representations: how representations evo…
Context Structure Reshapes the Representational Geometry of Language Models
Eghbal A. Hosseini, Yuxuan Li, Yasaman Bahri +2
Large Language Models (LLMs) have been shown to organize the representations of input sequences into straighter neural trajectories in their deep layers, which has been hypothesize…
SIMA 2: A Generalist Embodied Agent for Virtual Worlds
SIMA team, Adrian Bolton, Alexander Lerchner +63
We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a signifi…
Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences
Andrew Kyle Lampinen, Martin Engelcke, Yuxuan Li +2
When do machine learning systems fail to generalize, and what mechanisms could improve their generalization? Here, we draw inspiration from cognitive science to argue that one weak…
Just-in-time and distributed task representations in language models
Yuxuan Li, Declan Campbell, Stephanie C. Y. Chan +1
Many of language models' impressive capabilities originate from their in-context learning: based on instructions or examples, they can infer and perform new tasks without weight up…
Representation biases: will we achieve complete understanding by analyzing representations?
Andrew Kyle Lampinen, Stephanie C. Y. Chan, Yuxuan Li +1
A common approach in neuroscience is to study neural representations as a means to understand a system -- increasingly, by relating the neural representations to the internal repre…
Distinct Computations Emerge From Compositional Curricula in In-Context Learning
Jin Hwa Lee, Andrew K. Lampinen, Aaditya K. Singh +1
In-context learning (ICL) research often considers learning a function in-context through a uniform sample of input-output pairs. Here, we investigate how presenting a compositiona…
The emergence of sparse attention: impact of data distribution and benefits of repetition
Nicolas Zucchet, Francesco d'Angelo, Andrew K. Lampinen +1
Emergence is a fascinating property of large language models and neural networks more broadly: as models scale and train for longer, they sometimes develop new abilities in sudden…
On the generalization of language models from in-context learning and finetuning: a controlled study
Andrew K. Lampinen, Arslan Chaudhry, Stephanie C. Y. Chan +7
Large language models exhibit exciting capabilities, yet can show surprisingly narrow generalization from finetuning. E.g. they can fail to generalize to simple reversals of relati…
The in-context inductive biases of vision-language models differ across modalities
Kelsey Allen, Ishita Dasgupta, Eliza Kosoy +1
Inductive biases are what allow learners to make guesses in the absence of conclusive evidence. These biases have often been studied in cognitive science using concepts or categori…
Decoupling the components of geometric understanding in Vision Language Models
Eliza Kosoy, Annya Dahmani, Andrew K. Lampinen +4
Understanding geometry relies heavily on vision. In this work, we evaluate whether state-of-the-art vision language models (VLMs) can understand simple geometric concepts. We use a…
The broader spectrum of in-context learning
Andrew Kyle Lampinen, Stephanie C. Y. Chan, Aaditya K. Singh +1
The ability of language models to learn a task from a few examples in context has generated substantial interest. Here, we provide a perspective that situates this type of supervis…
Aligning Machine and Human Visual Representations across Abstraction Levels
Lukas Muttenthaler, Klaus Greff, Frieda Born +6
Deep neural networks have achieved success across a wide range of applications, including as models of human behavior and neural representations in vision tasks. However, neural ne…
Understanding Visual Feature Reliance through the Lens of Complexity
Thomas Fel, Louis Bethune, Andrew Kyle Lampinen +2
Recent studies suggest that deep learning models inductive bias towards favoring simpler features may be one of the sources of shortcut learning. Yet, there has been limited focus…
Learned feature representations are biased by complexity, learning order, position, and more
Andrew Kyle Lampinen, Stephanie C. Y. Chan, Katherine Hermann
Representation learning, and interpreting learned representations, are key areas of focus in machine learning and neuroscience. Both fields generally use representations as a means…
Scaling Instructable Agents Across Many Simulated Worlds
SIMA Team, Maria Abi Raad, Arun Ahuja +91
Building embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI. Accomplishing this goal requires lear…
SODA: Bottleneck Diffusion Models for Representation Learning
Drew A. Hudson, Daniel Zoran, Mateusz Malinowski +6
We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorporates an image encoder, which distills a source view into a compact rep…
Combining Behaviors with the Successor Features Keyboard
Wilka Carvalho, Andre Saraiva, Angelos Filos +7
The Option Keyboard (OK) was recently proposed as a method for transferring behavioral knowledge across tasks. OK transfers knowledge by adaptively combining subsets of known behav…
Evaluating Spatial Understanding of Large Language Models
Yutaro Yamada, Yihan Bao, Andrew K. Lampinen +2
Large language models (LLMs) show remarkable capabilities across a variety of tasks. Despite the models only seeing text in training, several recent studies suggest that LLM repres…
Getting aligned on representational alignment
Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller +30
Biological and artificial information processing systems form representations of the world that they can use to categorize, reason, plan, navigate, and make decisions. How can we m…