164 citations · 356 across the 41 of their papers we have counts for
44 papers
FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders
Hongyang Du, Yunfei Xie, Junjie Ye +13
Representation autoencoders (RAEs) reuse features from a pretrained visual encoder as reconstruction and diffusion latents, integrating strong visual representations into image gen…
Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules
Florian Rottach, Sebastian Schieferdecker, William Rudman +2
Despite recent advances in molecular foundation models, several limitations remain, such as chemically invalid augmentations, modality collapse, and incomplete representation of bi…
Self-supervised DXA representations encode multi-system disease risk, biological aging and heritability
Gil Sasson, Zachary Levine, Smadar Shilo +9
Whole-body dual-energy X-ray absorptiometry (DXA) scans are routinely acquired to measure bone density and regional body composition, leaving their spatial structure largely unused…
Semantic Tube Prediction: Beating LLM Data Efficiency with JEPA
Hai Huang, Yann LeCun, Randall Balestriero
Large Language Models (LLMs) obey consistent scaling laws -- empirical power-law fits that predict how loss decreases with compute, data, and parameters. While predictive, these la…
stable-worldmodel-v1: Reproducible World Modeling Research and Evaluation
Lucas Maes, Quentin Le Lidec, Dan Haramati +4
World Models have emerged as a powerful paradigm for learning compact, predictive representations of environment dynamics, enabling agents to reason, plan, and generalize beyond di…
Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization
Yilun Kuang, Yash Dagade, Deep Chakraborty +4
Self-supervised learning aims to learn maximally informative representations, but explicit information maximization is hindered by the curse of dimensionality. Existing methods lik…
Interpreting Physics in Video World Models
Sonia Joseph, Quentin Garrido, Randall Balestriero +5
A long-standing question in physical reasoning is whether video-based models need to rely on factorized representations of physical variables in order to make physically accurate p…
A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures
Basile Terver, Randall Balestriero, Megi Dervishi +8
We present EB-JEPA, an open-source library for learning representations and world models using Joint-Embedding Predictive Architectures (JEPAs). JEPAs learn to predict in represent…
Rectified LpJEPA: Joint-Embedding Predictive Architectures with Sparse and Maximum-Entropy Representations
Yilun Kuang, Yash Dagade, Tim G. J. Rudner +2
Joint-Embedding Predictive Architectures (JEPA) learn view-invariant representations and admit projection-based distribution matching for collapse prevention. Existing approaches r…
VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generation
Hongyang Du, Junjie Ye, Xiaoyan Cong +7
While recent video diffusion models (VDMs) produce visually impressive results, they fundamentally struggle to maintain 3D structural consistency, often resulting in object deforma…
OpenApps: Simulating Environment Variations to Measure UI-Agent Reliability
Karen Ullrich, Jingtong Su, Claudia Shi +7
Reliability is key to realizing the promise of autonomous UI-Agents, multimodal agents that directly interact with apps in the same manner as humans, as users must be able to trust…
LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics
Randall Balestriero, Yann LeCun
Learning manipulable representations of the world and its dynamics is central to AI. Joint-Embedding Predictive Architectures (JEPAs) offer a promising blueprint, but lack of pract…
Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density
Randall Balestriero, Nicolas Ballas, Mike Rabbat +1
Joint Embedding Predictive Architectures (JEPAs) learn representations able to solve numerous downstream tasks out-of-the-box. JEPAs combine two objectives: (i) a latent-space pred…
LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures
Hai Huang, Yann LeCun, Randall Balestriero
Large Language Model (LLM) pretraining, finetuning, and evaluation rely on input-space reconstruction and generative capabilities. Yet, it has been observed in vision that embeddin…
Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learning
Hugues Van Assel, Mark Ibrahim, Tommaso Biancalani +2
Reconstruction and joint embedding have emerged as two leading paradigms in Self Supervised Learning (SSL). Reconstruction methods focus on recovering the original sample from a di…
Ditch the Denoiser: Emergence of Noise Robustness in Self-Supervised Learning from Data Curriculum
Wenquan Lu, Jiaqi Zhang, Hugues Van Assel +1
Self-Supervised Learning (SSL) has become a powerful solution to extract rich representations from unlabeled data. Yet, SSL research is mostly focused on clean, curated and high-qu…
Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models
Vlad Sobal, Wancong Zhang, Kyunghyun Cho +3
A long-standing goal in AI is to develop agents capable of solving diverse tasks across a range of environments, including those never seen during training. Two dominant paradigms…
UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling
Haider Al-Tahan, Quentin Garrido, Randall Balestriero +3
Significant research efforts have been made to scale and improve vision-language model (VLM) training approaches. Yet, with an ever-growing number of benchmarks, researchers are ta…
-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs
Vlad Sobal, Mark Ibrahim, Randall Balestriero +5
Learning good representations involves capturing the diverse ways in which data samples relate. Contrastive loss - an objective matching related samples - underlies methods from se…
Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?
Mark Ibrahim, David Klindt, Randall Balestriero
Deep Learning is often depicted as a trio of data-architecture-loss. Yet, recent Self Supervised Learning (SSL) solutions have introduced numerous additional design choices, e.g.,…
Learning by Reconstruction Produces Uninformative Features For Perception
Randall Balestriero, Yann LeCun
Input space reconstruction is an attractive representation learning paradigm. Despite interpretability of the reconstruction and generation, we identify a misalignment between lear…
Fast and Exact Enumeration of Deep Networks Partitions Regions
Randall Balestriero, Yann LeCun
One fruitful formulation of Deep Networks (DNs) enabling their theoretical study and providing practical guidelines to practitioners relies on Piecewise Affine Splines. In that rea…
Understanding the Detrimental Class-level Effects of Data Augmentation
Polina Kirichenko, Mark Ibrahim, Randall Balestriero +4
Data augmentation (DA) encodes invariance and provides implicit regularization critical to a model's performance in image classification tasks. However, while DA improves average a…
A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation
Florian Bordes, Samuel Lavoie, Randall Balestriero +2
Self-Supervised Learning (SSL) models rely on a pretext task to learn representations. Because this pretext task differs from the downstream tasks used to evaluate the performance…