1.6k citations · 9.7k across the 222 of their papers we have counts for
35 papers · 1 filter
PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning
Yining Hong, Li Yi, Joshua B. Tenenbaum +2
A critical aspect of human visual perception is the ability to parse visual scenes into individual objects and further into object parts, forming part-whole hierarchies. Such compo…
Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
Anthony Simeonov, Yilun Du, Andrea Tagliasacchi +4
We present Neural Descriptor Fields (NDFs), an object representation that encodes both points and relative poses between an object and a target (such as a robot gripper or a rack u…
Noether Networks: Meta-Learning Useful Conserved Quantities
Ferran Alet, Dylan Doblar, Allan Zhou +3
Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often explo…
Learning to Compose Visual Relations
Nan Liu, Shuang Li, Yilun Du +2
The visual world around us can be described as a structured set of objects and their associated relations. An image of a room may be conjured given only the description of the unde…
Learning Signal-Agnostic Manifolds of Neural Fields
Yilun Du, Katherine M. Collins, Joshua B. Tenenbaum +1
Deep neural networks have been used widely to learn the latent structure of datasets, across modalities such as images, shapes, and audio signals. However, existing models are gene…
Unsupervised Learning of Compositional Energy Concepts
Yilun Du, Shuang Li, Yash Sharma +2
Humans are able to rapidly understand scenes by utilizing concepts extracted from prior experience. Such concepts are diverse, and include global scene descriptors, such as the wea…
MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation
Safa C. Medin, Bernhard Egger, Anoop Cherian +4
Recent advances in generative adversarial networks (GANs) have led to remarkable achievements in face image synthesis. While methods that use style-based GANs can generate striking…
3DP3: 3D Scene Perception via Probabilistic Programming
Nishad Gothoskar, Marco Cusumano-Towner, Ben Zinberg +6
We present 3DP3, a framework for inverse graphics that uses inference in a structured generative model of objects, scenes, and images. 3DP3 uses (i) voxel models to represent the 3…
Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language
Mingyu Ding, Zhenfang Chen, Tao Du +3
In this work, we propose a unified framework, called Visual Reasoning with Differ-entiable Physics (VRDP), that can jointly learn visual concepts and infer physics models of object…
OPEn: An Open-ended Physics Environment for Learning Without a Task
Chuang Gan, Abhishek Bhandwaldar, Antonio Torralba +2
Humans have mental models that allow them to plan, experiment, and reason in the physical world. How should an intelligent agent go about learning such models? In this paper, we wi…
Incorporating Rich Social Interactions Into MDPs
Ravi Tejwani, Yen-Ling Kuo, Tianmin Shu +5
Much of what we do as humans is engage socially with other agents, a skill that robots must also eventually possess. We demonstrate that a rich theory of social interactions origin…
Identity-Expression Ambiguity in 3D Morphable Face Models
Bernhard Egger, Skylar Sutherland, Safa C. Medin +1
3D Morphable Models are a class of generative models commonly used to model faces. They are typically applied to ill-posed problems such as 3D reconstruction from 2D data. Several…
Discovering State and Action Abstractions for Generalized Task and Motion Planning
Aidan Curtis, Tom Silver, Joshua B. Tenenbaum +2
Generalized planning accelerates classical planning by finding an algorithm-like policy that solves multiple instances of a task. A generalized plan can be learned from a few train…
Dynamic Modeling of Hand-Object Interactions via Tactile Sensing
Qiang Zhang, Yunzhu Li, Yiyue Luo +6
Tactile sensing is critical for humans to perform everyday tasks. While significant progress has been made in analyzing object grasping from vision, it remains unclear how we can u…
Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning
Pedro A. Tsividis, Joao Loula, Jake Burga +5
Reinforcement learning (RL) studies how an agent comes to achieve reward in an environment through interactions over time. Recent advances in machine RL have surpassed human expert…
Learning to solve complex tasks by growing knowledge culturally across generations
Michael Henry Tessler, Jason Madeano, Pedro A. Tsividis +3
Knowledge built culturally across generations allows humans to learn far more than an individual could glean from their own experience in a lifetime. Cultural knowledge in turn res…
Hybrid Memoised Wake-Sleep: Approximate Inference at the Discrete-Continuous Interface
Tuan Anh Le, Katherine M. Collins, Luke Hewitt +4
Modeling complex phenomena typically involves the use of both discrete and continuous variables. Such a setting applies across a wide range of problems, from identifying trends in…
Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning
Maxwell Nye, Michael Henry Tessler, Joshua B. Tenenbaum +1
Human reasoning can often be understood as an interplay between two systems: the intuitive and associative ("System 1") and the deliberative and logical ("System 2"). Neural sequen…
Modeling the Mistakes of Boundedly Rational Agents Within a Bayesian Theory of Mind
Arwa Alanqary, Gloria Z. Lin, Joie Le +3
When inferring the goals that others are trying to achieve, people intuitively understand that others might make mistakes along the way. This is crucial for activities such as teac…
Temporal and Object Quantification Networks
Jiayuan Mao, Zhezheng Luo, Chuang Gan +4
We present Temporal and Object Quantification Networks (TOQ-Nets), a new class of neuro-symbolic networks with a structural bias that enables them to learn to recognize complex rel…
Leveraging Language to Learn Program Abstractions and Search Heuristics
Catherine Wong, Kevin Ellis, Joshua B. Tenenbaum +1
Inductive program synthesis, or inferring programs from examples of desired behavior, offers a general paradigm for building interpretable, robust, and generalizable machine learni…
Physion: Evaluating Physical Prediction from Vision in Humans and Machines
Daniel M. Bear, Elias Wang, Damian Mrowca +12
While current vision algorithms excel at many challenging tasks, it is unclear how well they understand the physical dynamics of real-world environments. Here we introduce Physion,…
Communicating Natural Programs to Humans and Machines
Samuel Acquaviva, Yewen Pu, Marta Kryven +6
The Abstraction and Reasoning Corpus (ARC) is a set of procedural tasks that tests an agent's ability to flexibly solve novel problems. While most ARC tasks are easy for humans, th…
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering
Vincent Sitzmann, Semon Rezchikov, William T. Freeman +2
Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. Emerging 3D-structured neur…