98 citations · 851 across the 68 of their papers we have counts for
26 papers · 1 filter
Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion
Boyuan Chen, Diego Marti Monso, Yilun Du +3
This paper presents Diffusion Forcing, a new training paradigm where a diffusion model is trained to denoise a set of tokens with independent per-token noise levels. We apply Diffu…
Learning Iterative Reasoning through Energy Diffusion
Yilun Du, Jiayuan Mao, Joshua B. Tenenbaum
We introduce iterative reasoning through energy diffusion (IRED), a novel framework for learning to reason for a variety of tasks by formulating reasoning and decision-making probl…
Bridging Associative Memory and Probabilistic Modeling
Rylan Schaeffer, Nika Zahedi, Mikail Khona +9
Associative memory and probabilistic modeling are two fundamental topics in artificial intelligence. The first studies recurrent neural networks designed to denoise, complete and r…
Compositional Generative Modeling: A Single Model is Not All You Need
Yilun Du, Leslie Kaelbling
Large monolithic generative models trained on massive amounts of data have become an increasingly dominant approach in AI research. In this paper, we argue that we should instead c…
Inferring Relational Potentials in Interacting Systems
Armand Comas-Massagué, Yilun Du, Christian Fernandez +4
Systems consisting of interacting agents are prevalent in the world, ranging from dynamical systems in physics to complex biological networks. To build systems which can interact r…
Compositional Foundation Models for Hierarchical Planning
Anurag Ajay, Seungwook Han, Yilun Du +7
To make effective decisions in novel environments with long-horizon goals, it is crucial to engage in hierarchical reasoning across spatial and temporal scales. This entails planni…
Training Diffusion Models with Reinforcement Learning
Kevin Black, Michael Janner, Yilun Du +2
Diffusion models are a class of flexible generative models trained with an approximation to the log-likelihood objective. However, most use cases of diffusion models are not concer…
Planning with Sequence Models through Iterative Energy Minimization
Hongyi Chen, Yilun Du, Yiye Chen +2
Recent works have shown that sequence modeling can be effectively used to train reinforcement learning (RL) policies. However, the success of applying existing sequence models to p…
Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC
Yilun Du, Conor Durkan, Robin Strudel +6
Since their introduction, diffusion models have quickly become the prevailing approach to generative modeling in many domains. They can be interpreted as learning the gradients of…
Is Conditional Generative Modeling all you need for Decision-Making?
Anurag Ajay, Yilun Du, Abhi Gupta +3
Recent improvements in conditional generative modeling have made it possible to generate high-quality images from language descriptions alone. We investigate whether these methods…
Learning Iterative Reasoning through Energy Minimization
Yilun Du, Shuang Li, Joshua B. Tenenbaum +1
Deep learning has excelled on complex pattern recognition tasks such as image classification and object recognition. However, it struggles with tasks requiring nontrivial reasoning…
Streaming Inference for Infinite Non-Stationary Clustering
Rylan Schaeffer, Gabrielle Kaili-May Liu, Yilun Du +2
Learning from a continuous stream of non-stationary data in an unsupervised manner is arguably one of the most common and most challenging settings facing intelligent agents. Here,…
Planning with Diffusion for Flexible Behavior Synthesis
Michael Janner, Yilun Du, Joshua B. Tenenbaum +1
Model-based reinforcement learning methods often use learning only for the purpose of estimating an approximate dynamics model, offloading the rest of the decision-making work to c…
Pre-Trained Language Models for Interactive Decision-Making
Shuang Li, Xavier Puig, Chris Paxton +11
Language model (LM) pre-training is useful in many language processing tasks. But can pre-trained LMs be further leveraged for more general machine learning problems? We propose an…
Learning Signal-Agnostic Manifolds of Neural Fields
Yilun Du, Katherine M. Collins, Joshua B. Tenenbaum +1
Deep neural networks have been used widely to learn the latent structure of datasets, across modalities such as images, shapes, and audio signals. However, existing models are gene…
The Neural MMO Platform for Massively Multiagent Research
Joseph Suarez, Yilun Du, Clare Zhu +2
Neural MMO is a computationally accessible research platform that combines large agent populations, long time horizons, open-ended tasks, and modular game systems. Existing environ…
Improved Contrastive Divergence Training of Energy Based Models
Yilun Du, Shuang Li, Joshua Tenenbaum +1
Contrastive divergence is a popular method of training energy-based models, but is known to have difficulties with training stability. We propose an adaptation to improve contrasti…
Energy-Based Models for Continual Learning
Shuang Li, Yilun Du, Gido M. van de Ven +1
We motivate Energy-Based Models (EBMs) as a promising model class for continual learning problems. Instead of tackling continual learning via the use of external memory, growing mo…
Learning Object-Based State Estimators for Household Robots
Yilun Du, Tomas Lozano-Perez, Leslie Kaelbling
A robot operating in a household makes observations of multiple objects as it moves around over the course of days or weeks. The objects may be moved by inhabitants, but not comple…
Energy-based models for atomic-resolution protein conformations
Yilun Du, Joshua Meier, Jerry Ma +2
We propose an energy-based model (EBM) of protein conformations that operates at atomic scale. The model is trained solely on crystallized protein data. By contrast, existing appro…
Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks
Joseph Suarez, Yilun Du, Igor Mordatch +1
Progress in multiagent intelligence research is fundamentally limited by the number and quality of environments available for study. In recent years, simulated games have become a…
Observational Overfitting in Reinforcement Learning
Xingyou Song, Yiding Jiang, Stephen Tu +2
A major component of overfitting in model-free reinforcement learning (RL) involves the case where the agent may mistakenly correlate reward with certain spurious features from the…
Model Based Planning with Energy Based Models
Yilun Du, Toru Lin, Igor Mordatch
Model-based planning holds great promise for improving both sample efficiency and generalization in reinforcement learning (RL). We show that energy-based models (EBMs) are a promi…
An Empirical Study on Hyperparameters and their Interdependence for RL Generalization
Xingyou Song, Yilun Du, Jacob Jackson
Recent results in Reinforcement Learning (RL) have shown that agents with limited training environments are susceptible to a large amount of overfitting across many domains. A key…