108 citations · 415 across the 23 of their papers we have counts for
23 papers
Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks
Dan Yamins, Aran Nayebi
A series of results from the NeuroAI over the past fifteen years have raised core questions both about how to compare Deep Neural Network (DNN) models to the brain, and about how m…
Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
Logan Cross, Violet Xiang, Agam Bhatia +2
Multi-agent reinforcement learning (MARL) methods struggle with the non-stationarity of multi-agent systems and fail to adaptively learn online when tested with novel agents. Here,…
Unifying (Machine) Vision via Counterfactual World Modeling
Daniel M. Bear, Kevin Feigelis, Honglin Chen +5
Leading approaches in machine vision employ different architectures for different tasks, trained on costly task-specific labeled datasets. This complexity has held back progress in…
Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties
Hsiao-Yu Tung, Mingyu Ding, Zhenfang Chen +6
General physical scene understanding requires more than simply localizing and recognizing objects -- it requires knowledge that objects can have different latent properties (e.g.,…
Developmental Curiosity and Social Interaction in Virtual Agents
Chris Doyle, Sarah Shader, Michelle Lau +3
Infants explore their complex physical and social environment in an organized way. To gain insight into what intrinsic motivations may help structure this exploration, we create a…
Measuring and Modeling Physical Intrinsic Motivation
Julio Martinez, Felix Binder, Haoliang Wang +3
Humans are interactive agents driven to seek out situations with interesting physical dynamics. Here we formalize the functional form of physical intrinsic motivation. We first col…
3D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenes
Haotian Xue, Antonio Torralba, Joshua B. Tenenbaum +3
Given a visual scene, humans have strong intuitions about how a scene can evolve over time under given actions. The intuition, often termed visual intuitive physics, is a critical…
Unsupervised Segmentation in Real-World Images via Spelke Object Inference
Honglin Chen, Rahul Venkatesh, Yoni Friedman +4
Self-supervised, category-agnostic segmentation of real-world images is a challenging open problem in computer vision. Here, we show how to learn static grouping priors from motion…
The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AI
Chuang Gan, Siyuan Zhou, Jeremy Schwartz +8
We introduce a visually-guided and physics-driven task-and-motion planning benchmark, which we call the ThreeDWorld Transport Challenge. In this challenge, an embodied agent equipp…
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics
Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli +2
Understanding the dynamics of neural network parameters during training is one of the key challenges in building a theoretical foundation for deep learning. A central obstacle is t…
Conditional Negative Sampling for Contrastive Learning of Visual Representations
Mike Wu, Milan Mosse, Chengxu Zhuang +2
Recent methods for learning unsupervised visual representations, dubbed contrastive learning, optimize the noise-contrastive estimation (NCE) bound on mutual information between tw…
Identifying Learning Rules From Neural Network Observables
Aran Nayebi, Sanjana Srivastava, Surya Ganguli +1
The brain modifies its synaptic strengths during learning in order to better adapt to its environment. However, the underlying plasticity rules that govern learning are unknown. Ma…
Active World Model Learning with Progress Curiosity
Kuno Kim, Megumi Sano, Julian De Freitas +2
World models are self-supervised predictive models of how the world evolves. Humans learn world models by curiously exploring their environment, in the process acquiring compact ab…
Learning Physical Graph Representations from Visual Scenes
Daniel M. Bear, Chaofei Fan, Damian Mrowca +8
Convolutional Neural Networks (CNNs) have proved exceptional at learning representations for visual object categorization. However, CNNs do not explicitly encode objects, parts, an…
On Mutual Information in Contrastive Learning for Visual Representations
Mike Wu, Chengxu Zhuang, Milan Mosse +2
In recent years, several unsupervised, "contrastive" learning algorithms in vision have been shown to learn representations that perform remarkably well on transfer tasks. We show…
Pruning neural networks without any data by iteratively conserving synaptic flow
Hidenori Tanaka, Daniel Kunin, Daniel L. K. Yamins +1
Pruning the parameters of deep neural networks has generated intense interest due to potential savings in time, memory and energy both during training and at test time. Recent work…
Visual Grounding of Learned Physical Models
Yunzhu Li, Toru Lin, Kexin Yi +5
Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions. The abilities to perform physical…
Flexible and Efficient Long-Range Planning Through Curious Exploration
Aidan Curtis, Minjian Xin, Dilip Arumugam +2
Identifying algorithms that flexibly and efficiently discover temporally-extended multi-phase plans is an essential step for the advancement of robotics and model-based reinforceme…
Two Routes to Scalable Credit Assignment without Weight Symmetry
Daniel Kunin, Aran Nayebi, Javier Sagastuy-Brena +3
The neural plausibility of backpropagation has long been disputed, primarily for its use of non-local weight transport the biologically dubious requirement that one neuron inst…
Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs
Jonas Kubilius, Martin Schrimpf, Kohitij Kar +11
Deep convolutional artificial neural networks (ANNs) are the leading class of candidate models of the mechanisms of visual processing in the primate ventral stream. While initially…
Emergence of Structured Behaviors from Curiosity-Based Intrinsic Motivation
Nick Haber, Damian Mrowca, Li Fei-Fei +1
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to rep…
Modular Continual Learning in a Unified Visual Environment
Kevin T. Feigelis, Blue Sheffer, Daniel L. K. Yamins
A core aspect of human intelligence is the ability to learn new tasks quickly and switch between them flexibly. Here, we describe a modular continual reinforcement learning paradig…
A Useful Motif for Flexible Task Learning in an Embodied Two-Dimensional Visual Environment
Kevin T. Feigelis, Daniel L. K. Yamins
Animals (especially humans) have an amazing ability to learn new tasks quickly, and switch between them flexibly. How brains support this ability is largely unknown, both neuroscie…