1.6k citations · 9.7k across the 222 of their papers we have counts for
46 papers · 1 filter
Inferring Relational Potentials in Interacting Systems
Armand Comas-Massagué, Yilun Du, Christian Fernandez +4
Systems consisting of interacting agents are prevalent in the world, ranging from dynamical systems in physics to complex biological networks. To build systems which can interact r…
Evaluating Language Models for Mathematics through Interactions
Katherine M. Collins, Albert Q. Jiang, Simon Frieder +11
There is much excitement about the opportunity to harness the power of large language models (LLMs) when building problem-solving assistants. However, the standard methodology of e…
Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics
Pingchuan Ma, Peter Yichen Chen, Bolei Deng +4
We propose a hybrid neural network (NN) and PDE approach for learning generalizable PDE dynamics from motion observations. Many NN approaches learn an end-to-end model that implici…
Planning with Sequence Models through Iterative Energy Minimization
Hongyi Chen, Yilun Du, Yiye Chen +2
Recent works have shown that sequence modeling can be effectively used to train reinforcement learning (RL) policies. However, the success of applying existing sequence models to p…
Planning with Large Language Models for Code Generation
Shun Zhang, Zhenfang Chen, Yikang Shen +3
Existing large language model-based code generation pipelines typically use beam search or sampling algorithms during the decoding process. Although the programs they generate achi…
On the Expressiveness and Generalization of Hypergraph Neural Networks
Zhezheng Luo, Jiayuan Mao, Joshua B. Tenenbaum +1
This extended abstract describes a framework for analyzing the expressiveness, learning, and (structural) generalization of hypergraph neural networks (HyperGNNs). Specifically, we…
Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC
Yilun Du, Conor Durkan, Robin Strudel +6
Since their introduction, diffusion models have quickly become the prevailing approach to generative modeling in many domains. They can be interpreted as learning the gradients of…
Discovering Generalizable Spatial Goal Representations via Graph-based Active Reward Learning
Aviv Netanyahu, Tianmin Shu, Joshua Tenenbaum +1
In this work, we consider one-shot imitation learning for object rearrangement tasks, where an AI agent needs to watch a single expert demonstration and learn to perform the same t…
Is Conditional Generative Modeling all you need for Decision-Making?
Anurag Ajay, Yilun Du, Abhi Gupta +3
Recent improvements in conditional generative modeling have made it possible to generate high-quality images from language descriptions alone. We investigate whether these methods…
Learning Physical Dynamics with Subequivariant Graph Neural Networks
Jiaqi Han, Wenbing Huang, Hengbo Ma +3
Graph Neural Networks (GNNs) have become a prevailing tool for learning physical dynamics. However, they still encounter several challenges: 1) Physical laws abide by symmetry, whi…
Learning Iterative Reasoning through Energy Minimization
Yilun Du, Shuang Li, Joshua B. Tenenbaum +1
Deep learning has excelled on complex pattern recognition tasks such as image classification and object recognition. However, it struggles with tasks requiring nontrivial reasoning…
Prompting Decision Transformer for Few-Shot Policy Generalization
Mengdi Xu, Yikang Shen, Shun Zhang +4
Humans can leverage prior experience and learn novel tasks from a handful of demonstrations. In contrast to offline meta-reinforcement learning, which aims to achieve quick adaptat…
Planning with Diffusion for Flexible Behavior Synthesis
Michael Janner, Yilun Du, Joshua B. Tenenbaum +1
Model-based reinforcement learning methods often use learning only for the purpose of estimating an approximate dynamics model, offloading the rest of the decision-making work to c…
DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools
Xingyu Lin, Zhiao Huang, Yunzhu Li +3
We consider the problem of sequential robotic manipulation of deformable objects using tools. Previous works have shown that differentiable physics simulators provide gradients to…
Noether Networks: Meta-Learning Useful Conserved Quantities
Ferran Alet, Dylan Doblar, Allan Zhou +3
Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often explo…
Learning Signal-Agnostic Manifolds of Neural Fields
Yilun Du, Katherine M. Collins, Joshua B. Tenenbaum +1
Deep neural networks have been used widely to learn the latent structure of datasets, across modalities such as images, shapes, and audio signals. However, existing models are gene…
Temporal and Object Quantification Networks
Jiayuan Mao, Zhezheng Luo, Chuang Gan +4
We present Temporal and Object Quantification Networks (TOQ-Nets), a new class of neuro-symbolic networks with a structural bias that enables them to learn to recognize complex rel…
Leveraging Language to Learn Program Abstractions and Search Heuristics
Catherine Wong, Kevin Ellis, Joshua B. Tenenbaum +1
Inductive program synthesis, or inferring programs from examples of desired behavior, offers a general paradigm for building interpretable, robust, and generalizable machine learni…
PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics
Zhiao Huang, Yuanming Hu, Tao Du +4
Simulated virtual environments serve as one of the main driving forces behind developing and evaluating skill learning algorithms. However, existing environments typically only sim…
Learning Task Decomposition with Ordered Memory Policy Network
Yuchen Lu, Yikang Shen, Siyuan Zhou +3
Many complex real-world tasks are composed of several levels of sub-tasks. Humans leverage these hierarchical structures to accelerate the learning process and achieve better gener…
Augmenting Policy Learning with Routines Discovered from a Single Demonstration
Zelin Zhao, Chuang Gan, Jiajun Wu +2
Humans can abstract prior knowledge from very little data and use it to boost skill learning. In this paper, we propose routine-augmented policy learning (RAPL), which discovers ro…
Improved Contrastive Divergence Training of Energy Based Models
Yilun Du, Shuang Li, Joshua Tenenbaum +1
Contrastive divergence is a popular method of training energy-based models, but is known to have difficulties with training stability. We propose an adaptation to improve contrasti…
Data-Efficient Learning for Complex and Real-Time Physical Problem Solving using Augmented Simulation
Kei Ota, Devesh K. Jha, Diego Romeres +7
Humans quickly solve tasks in novel systems with complex dynamics, without requiring much interaction. While deep reinforcement learning algorithms have achieved tremendous success…
Planning with Learned Object Importance in Large Problem Instances using Graph Neural Networks
Tom Silver, Rohan Chitnis, Aidan Curtis +3
Real-world planning problems often involve hundreds or even thousands of objects, straining the limits of modern planners. In this work, we address this challenge by learning to pr…