4 citations · 6 across the 10 of their papers we have counts for
10 papers
A Generalization Theory for JEPA-Based World Models
Jingyi Cui, Qi Zhang, Hongwei Wen +2
Joint Embedding Predictive Architectures (JEPAs) have emerged as a promising paradigm for world modeling by predicting future representations rather than reconstructing observation…
A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minima
Yiming Tang, Harshvardhan Saini, Zhaoqian Yao +6
As AI models achieve remarkable capabilities across diverse domains, understanding what representations they learn and how they encode concepts has become increasingly important fo…
On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedy
Jingyi Cui, Qi Zhang, Yifei Wang +1
Sparse autoencoders (SAEs) have recently emerged as a powerful tool for interpreting the features learned by large language models (LLMs). By reconstructing features with sparsely…
An Augmentation-Aware Theory for Self-Supervised Contrastive Learning
Jingyi Cui, Hongwei Wen, Yisen Wang
Self-supervised contrastive learning has emerged as a powerful tool in machine learning and computer vision to learn meaningful representations from unlabeled data. Meanwhile, its…
An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noise
Jingyi Cui, Yi-Ge Zhang, Hengyu Liu +1
Learning from noisy labels is a critical challenge in machine learning, with vast implications for numerous real-world scenarios. While supervised contrastive learning has recently…
Difficult Examples Hurt Unsupervised Contrastive Learning: A Theoretical Perspective
Yi-Ge Zhang, Jingyi Cui, Qiran Li +1
Unsupervised contrastive learning has shown significant performance improvements in recent years, often approaching or even rivaling supervised learning in various tasks. However,…
Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness
Qi Zhang, Yifei Wang, Jingyi Cui +4
Deep learning models often suffer from a lack of interpretability due to polysemanticity, where individual neurons are activated by multiple unrelated semantics, resulting in uncle…
Rethinking Weak Supervision in Helping Contrastive Learning
Jingyi Cui, Weiran Huang, Yifei Wang +1
Contrastive learning has shown outstanding performances in both supervised and unsupervised learning, and has recently been introduced to solve weakly supervised learning problems…
GBHT: Gradient Boosting Histogram Transform for Density Estimation
Jingyi Cui, Hanyuan Hang, Yisen Wang +1
In this paper, we propose a density estimation algorithm called \textit{Gradient Boosting Histogram Transform} (GBHT), where we adopt the \textit{Negative Log Likelihood} as the lo…
Leveraged Weighted Loss for Partial Label Learning
Hongwei Wen, Jingyi Cui, Hanyuan Hang +3
As an important branch of weakly supervised learning, partial label learning deals with data where each instance is assigned with a set of candidate labels, whereas only one of the…