7 citations · 25 across the 37 of their papers we have counts for
20 papers · 1 filter
Preserving Unstable Modes Through Inverse Dynamics in JEPA World Models
Leonardo F. Toso, Yann LeCun, James Anderson +1
Robotic systems often exhibit unstable modes, along which small perturbations and disturbances can cause unbounded growth unless corrected through feedback. Controlling such system…
H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
Wancong Zhang, Basile Terver, Michael Rabbat +2
Long-horizon planning with latent world models requires reasoning across timescales and levels of abstraction. Existing task-agnostic JEPA world models predict and plan at a single…
LpWM: A Case for Sparse Representations in World Models
Yilun Kuang, Yash Dagade, Quentin Le Lidec +3
Joint-embedding predictive architectures (JEPAs) learn latent dynamics for planning and avoid representation collapse by matching features to maximum-entropy distributions such as…
AdaJEPA: An Adaptive Latent World Model
Ying Wang, Oumayma Bounou, Yann LeCun +1
Latent world models enable planning from high-dimensional observations by predicting future states in a compact latent space. However, these models are typically kept frozen at tes…
stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation
Lucas Maes, Quentin Le Lidec, Luiz Facury +9
World models are central to building agents that can reason, plan, and generalize beyond their training data. However, research on world models is currently fragmented, with dispar…
On Training in Imagination
Nadav Timor, Ravid Shwartz-Ziv, Micah Goldblum +2
State-of-the-art model-based reinforcement learning methods train policies on imagined rollouts. These rollouts are trajectories generated by a learned dynamics model and are score…
Hierarchical Planning with Latent World Models
Wancong Zhang, Basile Terver, Artem Zholus +8
World models are a promising path to zero-shot embodied control through planning. However, existing world model planners struggle on long-horizon, multi-stage tasks: prediction err…
Representation Learning for Spatiotemporal Physical Systems
Helen Qu, Rudy Morel, Michael McCabe +4
Machine learning approaches to spatiotemporal physical systems have primarily focused on next-frame prediction, with the goal of learning an accurate emulator for the system's evol…
LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
Lucas Maes, Quentin Le Lidec, Damien Scieur +2
Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces, yet existing methods remain fragile, relying on co…
Temporal Straightening for Latent Planning
Ying Wang, Oumayma Bounou, Gaoyue Zhou +4
Learning good representations is essential for latent planning with world models. While pretrained visual encoders produce strong semantic visual features, they are not tailored to…
Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization
Yilun Kuang, Yash Dagade, Deep Chakraborty +4
Self-supervised learning aims to learn maximally informative representations, but explicit information maximization is hindered by the curse of dimensionality. Existing methods lik…
Rectified LpJEPA: Joint-Embedding Predictive Architectures with Sparse and Maximum-Entropy Representations
Yilun Kuang, Yash Dagade, Tim G. J. Rudner +2
Joint-Embedding Predictive Architectures (JEPA) learn view-invariant representations and admit projection-based distribution matching for collapse prevention. Existing approaches r…
Parallel Stochastic Gradient-Based Planning for World Models
Michael Psenka, Michael Rabbat, Aditi Krishnapriyan +2
World models simulate environment dynamics from raw sensory inputs like video. However, using them for planning can be challenging due to the vast and unstructured search space. We…
Value-guided action planning with JEPA world models
Matthieu Destrade, Oumayma Bounou, Quentin Le Lidec +2
Building deep learning models that can reason about their environment requires capturing its underlying dynamics. Joint-Embedded Predictive Architectures (JEPA) provide a promising…
Closing the Train-Test Gap in World Models for Gradient-Based Planning
Arjun Parthasarathy, Nimit Kalra, Rohun Agrawal +4
World models paired with model predictive control (MPC) can be trained offline on large-scale datasets of expert trajectories and enable generalization to a wide range of planning…
LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics
Randall Balestriero, Yann LeCun
Learning manipulable representations of the world and its dynamics is central to AI. Joint-Embedding Predictive Architectures (JEPAs) offer a promising blueprint, but lack of pract…
Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density
Randall Balestriero, Nicolas Ballas, Mike Rabbat +1
Joint Embedding Predictive Architectures (JEPAs) learn representations able to solve numerous downstream tasks out-of-the-box. JEPAs combine two objectives: (i) a latent-space pred…
Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models
Vlad Sobal, Wancong Zhang, Kyunghyun Cho +3
A long-standing goal in AI is to develop agents capable of solving diverse tasks across a range of environments, including those never seen during training. Two dominant paradigms…
Hierarchical World Models as Visual Whole-Body Humanoid Controllers
Nicklas Hansen, Jyothir S, Vlad Sobal +3
Whole-body control for humanoids is challenging due to the high-dimensional nature of the problem, coupled with the inherent instability of a bipedal morphology. Learning from visu…
Gradient-based Planning with World Models
Jyothir S, Siddhartha Jalagam, Yann LeCun +1
The enduring challenge in the field of artificial intelligence has been the control of systems to achieve desired behaviours. While for systems governed by straightforward dynamics…