1 citations · 2 across the 15 of their papers we have counts for
11 papers · 1 filter
A Generalization Theory for JEPA-Based World Models
Jingyi Cui, Qi Zhang, Hongwei Wen +2
Joint Embedding Predictive Architectures (JEPAs) have emerged as a promising paradigm for world modeling by predicting future representations rather than reconstructing observation…
Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
Tianqi Du, Qi Zhang, Yifei Wang +1
Recently, world models have emerged as a promising paradigm for building intelligent agents by learning predictive models that estimate future environment states conditioned on obs…
Leveraging Data Symmetries to Select an Optimal Subset of Training Data under Label Noise
Kumar Shubham, Pavan Karjol, Kiran M K +1
The performance of machine learning models often relies on large labeled datasets; however, data collected from diverse sources can contain label noise. Recent work has shown that,…
A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minima
Yiming Tang, Harshvardhan Saini, Zhaoqian Yao +6
As AI models achieve remarkable capabilities across diverse domains, understanding what representations they learn and how they encode concepts has become increasingly important fo…
On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedy
Jingyi Cui, Qi Zhang, Yifei Wang +1
Sparse autoencoders (SAEs) have recently emerged as a powerful tool for interpreting the features learned by large language models (LLMs). By reconstructing features with sparsely…
An Augmentation-Aware Theory for Self-Supervised Contrastive Learning
Jingyi Cui, Hongwei Wen, Yisen Wang
Self-supervised contrastive learning has emerged as a powerful tool in machine learning and computer vision to learn meaningful representations from unlabeled data. Meanwhile, its…
An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noise
Jingyi Cui, Yi-Ge Zhang, Hengyu Liu +1
Learning from noisy labels is a critical challenge in machine learning, with vast implications for numerous real-world scenarios. While supervised contrastive learning has recently…
Difficult Examples Hurt Unsupervised Contrastive Learning: A Theoretical Perspective
Yi-Ge Zhang, Jingyi Cui, Qiran Li +1
Unsupervised contrastive learning has shown significant performance improvements in recent years, often approaching or even rivaling supervised learning in various tasks. However,…
On the Role of Discrete Tokenization in Visual Representation Learning
Tianqi Du, Yifei Wang, Yisen Wang
In the realm of self-supervised learning (SSL), masked image modeling (MIM) has gained popularity alongside contrastive learning methods. MIM involves reconstructing masked regions…
Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining
Qi Zhang, Tianqi Du, Haotian Huang +2
In recent years, the rise of generative self-supervised learning (SSL) paradigms has exhibited impressive performance across visual, language, and multi-modal domains. While the va…
Non-negative Contrastive Learning
Yifei Wang, Qi Zhang, Yaoyu Guo +1
Deep representations have shown promising performance when transferred to downstream tasks in a black-box manner. Yet, their inherent lack of interpretability remains a significant…