143 citations · 440 across the 42 of their papers we have counts for
14 papers · 1 filter
Learning adaptive planning representations with natural language guidance
Lionel Wong, Jiayuan Mao, Pratyusha Sharma +5
Effective planning in the real world requires not only world knowledge, but the ability to leverage that knowledge to build the right representation of the task at hand. Decades of…
What Planning Problems Can A Relational Neural Network Solve?
Jiayuan Mao, Tomás Lozano-Pérez, Joshua B. Tenenbaum +1
Goal-conditioned policies are generally understood to be "feed-forward" circuits, in the form of neural networks that map from the current state and the goal specification to the n…
Learning Reusable Manipulation Strategies
Jiayuan Mao, Joshua B. Tenenbaum, Tomás Lozano-Pérez +1
Humans demonstrate an impressive ability to acquire and generalize manipulation "tricks." Even from a single demonstration, such as using soup ladles to reach for distant objects,…
What's Left? Concept Grounding with Logic-Enhanced Foundation Models
Joy Hsu, Jiayuan Mao, Joshua B. Tenenbaum +1
Recent works such as VisProg and ViperGPT have smartly composed foundation models for visual reasoning-using large language models (LLMs) to produce programs that can be executed b…
Learning to Act from Actionless Videos through Dense Correspondences
Po-Chen Ko, Jiayuan Mao, Yilun Du +2
In this work, we present an approach to construct a video-based robot policy capable of reliably executing diverse tasks across different robots and environments from few video dem…
HandMeThat: Human-Robot Communication in Physical and Social Environments
Yanming Wan, Jiayuan Mao, Joshua B. Tenenbaum
We introduce HandMeThat, a benchmark for a holistic evaluation of instruction understanding and following in physical and social environments. While previous datasets primarily foc…
CLEVRER-Humans: Describing Physical and Causal Events the Human Way
Jiayuan Mao, Xuelin Yang, Xikun Zhang +2
Building machines that can reason about physical events and their causal relationships is crucial for flexible interaction with the physical world. However, most existing physical…
Compositional Diffusion-Based Continuous Constraint Solvers
Zhutian Yang, Jiayuan Mao, Yilun Du +4
This paper introduces an approach for learning to solve continuous constraint satisfaction problems (CCSP) in robotic reasoning and planning. Previous methods primarily rely on han…
Programmatically Grounded, Compositionally Generalizable Robotic Manipulation
Renhao Wang, Jiayuan Mao, Joy Hsu +3
Robots operating in the real world require both rich manipulation skills as well as the ability to semantically reason about when to apply those skills. Towards this goal, recent w…
NS3D: Neuro-Symbolic Grounding of 3D Objects and Relations
Joy Hsu, Jiayuan Mao, Jiajun Wu
Grounding object properties and relations in 3D scenes is a prerequisite for a wide range of artificial intelligence tasks, such as visually grounded dialogues and embodied manipul…
Sparse and Local Networks for Hypergraph Reasoning
Guangxuan Xiao, Leslie Pack Kaelbling, Jiajun Wu +1
Reasoning about the relationships between entities from input facts (e.g., whether Ari is a grandparent of Charlie) generally requires explicit consideration of other entities that…
On the Expressiveness and Generalization of Hypergraph Neural Networks
Zhezheng Luo, Jiayuan Mao, Joshua B. Tenenbaum +1
This extended abstract describes a framework for analyzing the expressiveness, learning, and (structural) generalization of hypergraph neural networks (HyperGNNs). Specifically, we…
Learning Rational Subgoals from Demonstrations and Instructions
Zhezheng Luo, Jiayuan Mao, Jiajun Wu +3
We present a framework for learning useful subgoals that support efficient long-term planning to achieve novel goals. At the core of our framework is a collection of rational subgo…
PDSketch: Integrated Planning Domain Programming and Learning
Jiayuan Mao, Tomás Lozano-Pérez, Joshua B. Tenenbaum +1
This paper studies a model learning and online planning approach towards building flexible and general robots. Specifically, we investigate how to exploit the locality and sparsity…