1.6k citations · 9.7k across the 222 of their papers we have counts for
24 papers · 1 filter
Look, Listen, and Act: Towards Audio-Visual Embodied Navigation
Chuang Gan, Yiwei Zhang, Jiajun Wu +2
A crucial ability of mobile intelligent agents is to integrate the evidence from multiple sensory inputs in an environment and to make a sequence of actions to reach their goals. I…
Accurate Vision-based Manipulation through Contact Reasoning
Alina Kloss, Maria Bauza, Jiajun Wu +3
Planning contact interactions is one of the core challenges of many robotic tasks. Optimizing contact locations while taking dynamics into account is computationally costly and, in…
Entity Abstraction in Visual Model-Based Reinforcement Learning
Rishi Veerapaneni, John D. Co-Reyes, Michael Chang +5
This paper tests the hypothesis that modeling a scene in terms of entities and their local interactions, as opposed to modeling the scene globally, provides a significant benefit i…
CLEVRER: CoLlision Events for Video REpresentation and Reasoning
Kexin Yi, Chuang Gan, Yunzhu Li +4
The ability to reason about temporal and causal events from videos lies at the core of human intelligence. Most video reasoning benchmarks, however, focus on pattern recognition fr…
Program-Guided Image Manipulators
Jiayuan Mao, Xiuming Zhang, Yikai Li +3
Humans are capable of building holistic representations for images at various levels, from local objects, to pairwise relations, to global structures. The interpretation of structu…
DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs
Yunbo Wang, Bo Liu, Jiajun Wu +4
A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived…
Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning
Kelsey R. Allen, Kevin A. Smith, Joshua B. Tenenbaum
Many animals, and an increasing number of artificial agents, display sophisticated capabilities to perceive and manipulate objects. But human beings remain distinctive in their cap…
DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions
Zhenjia Xu, Jiajun Wu, Andy Zeng +2
We study the problem of learning physical object representations for robot manipulation. Understanding object physics is critical for successful object manipulation, but also chall…
Write, Execute, Assess: Program Synthesis with a REPL
Kevin Ellis, Maxwell Nye, Yewen Pu +3
We present a neural program synthesis approach integrating components which write, execute, and assess code to navigate the search space of possible programs. We equip the search p…
Finding Friend and Foe in Multi-Agent Games
Jack Serrino, Max Kleiman-Weiner, David C. Parkes +1
Recent breakthroughs in AI for multi-agent games like Go, Poker, and Dota, have seen great strides in recent years. Yet none of these games address the real-life challenge of coope…
Neurally-Guided Structure Inference
Sidi Lu, Jiayuan Mao, Joshua B. Tenenbaum +1
Most structure inference methods either rely on exhaustive search or are purely data-driven. Exhaustive search robustly infers the structure of arbitrarily complex data, but it is…
Explaining intuitive difficulty judgments by modeling physical effort and risk
Ilker Yildirim, Basil Saeed, Grace Bennett-Pierre +3
The ability to estimate task difficulty is critical for many real-world decisions such as setting appropriate goals for ourselves or appreciating others' accomplishments. Here we g…
The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
Jiayuan Mao, Chuang Gan, Pushmeet Kohli +2
We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that learns visual concepts, words, and semantic parsing of sentences without explicit supervision on any of them; in…
Combining Physical Simulators and Object-Based Networks for Control
Anurag Ajay, Maria Bauza, Jiajun Wu +4
Physics engines play an important role in robot planning and control; however, many real-world control problems involve complex contact dynamics that cannot be characterized analyt…
Few-Shot Bayesian Imitation Learning with Logical Program Policies
Tom Silver, Kelsey R. Allen, Alex K. Lew +2
Humans can learn many novel tasks from a very small number (1--5) of demonstrations, in stark contrast to the data requirements of nearly tabula rasa deep learning methods. We prop…
Unsupervised Discovery of Parts, Structure, and Dynamics
Zhenjia Xu, Zhijian Liu, Chen Sun +4
Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future. In this paper, we propose…
Stochastic Prediction of Multi-Agent Interactions from Partial Observations
Chen Sun, Per Karlsson, Jiajun Wu +2
We present a method that learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of…
Infinite Mixture Prototypes for Few-Shot Learning
Kelsey R. Allen, Evan Shelhamer, Hanul Shin +1
We propose infinite mixture prototypes to adaptively represent both simple and complex data distributions for few-shot learning. Our infinite mixture prototypes represent each clas…
Learning to Infer Program Sketches
Maxwell Nye, Luke Hewitt, Joshua Tenenbaum +1
Our goal is to build systems which write code automatically from the kinds of specifications humans can most easily provide, such as examples and natural language instruction. The…
The Omniglot challenge: a 3-year progress report
Brenden M. Lake, Ruslan Salakhutdinov, Joshua B. Tenenbaum
Three years ago, we released the Omniglot dataset for one-shot learning, along with five challenge tasks and a computational model that addresses these tasks. The model was not mea…
Theory of Minds: Understanding Behavior in Groups Through Inverse Planning
Michael Shum, Max Kleiman-Weiner, Michael L. Littman +1
Human social behavior is structured by relationships. We form teams, groups, tribes, and alliances at all scales of human life. These structures guide multi-agent cooperation and c…
Reasoning About Physical Interactions with Object-Oriented Prediction and Planning
Michael Janner, Sergey Levine, William T. Freeman +3
Object-based factorizations provide a useful level of abstraction for interacting with the world. Building explicit object representations, however, often requires supervisory sign…
On the Units of GANs (Extended Abstract)
David Bau, Jun-Yan Zhu, Hendrik Strobelt +4
Generative Adversarial Networks (GANs) have achieved impressive results for many real-world applications. As an active research topic, many GAN variants have emerged with improveme…
Learning to Infer and Execute 3D Shape Programs
Yonglong Tian, Andrew Luo, Xingyuan Sun +4
Human perception of 3D shapes goes beyond reconstructing them as a set of points or a composition of geometric primitives: we also effortlessly understand higher-level shape struct…