43 citations · 177 across the 17 of their papers we have counts for
10 papers · 1 filter
Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modeling
Xiao Li, Zekai Zhang, Xiang Li +4
Diffusion models, though originally designed for generative tasks, have demonstrated impressive self-supervised representation learning capabilities. A particularly intriguing phen…
Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination
Peng Wang, Xiao Li, Can Yaras +4
Over the past decade, deep learning has proven to be a highly effective tool for learning meaningful features from raw data. However, it remains an open question how deep networks…
A Provable Splitting Approach for Symmetric Nonnegative Matrix Factorization
Xiao Li, Zhihui Zhu, Qiuwei Li +1
The symmetric Nonnegative Matrix Factorization (NMF), a special but important class of the general NMF, has found numerous applications in data analysis such as various clustering…
Understanding and Improving Transfer Learning of Deep Models via Neural Collapse
Xiao Li, Sheng Liu, Jinxin Zhou +4
With the ever-increasing complexity of large-scale pre-trained models coupled with a shortage of labeled data for downstream training, transfer learning has become the primary appr…
Are All Losses Created Equal: A Neural Collapse Perspective
Jinxin Zhou, Chong You, Xiao Li +4
While cross entropy (CE) is the most commonly used loss to train deep neural networks for classification tasks, many alternative losses have been developed to obtain better empiric…
On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features
Jinxin Zhou, Xiao Li, Tianyu Ding +3
When training deep neural networks for classification tasks, an intriguing empirical phenomenon has been widely observed in the last-layer classifiers and features, where (i) the c…
A Geometric Analysis of Neural Collapse with Unconstrained Features
Zhihui Zhu, Tianyu Ding, Jinxin Zhou +4
We provide the first global optimization landscape analysis of -- an intriguing empirical phenomenon that arises in the last-layer classifiers and features of ne…
Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications
Qing Qu, Zhihui Zhu, Xiao Li +3
The problem of finding the sparsest vector (direction) in a low dimensional subspace can be considered as a homogeneous variant of the sparse recovery problem, which finds applicat…
Analysis of the Optimization Landscapes for Overcomplete Representation Learning
Qing Qu, Yuexiang Zhai, Xiao Li +2
We study nonconvex optimization landscapes for learning overcomplete representations, including learning (i) sparsely used overcomplete dictionaries and (ii) convolutional dictiona…
Dropping Symmetry for Fast Symmetric Nonnegative Matrix Factorization
Zhihui Zhu, Xiao Li, Kai Liu +1
Symmetric nonnegative matrix factorization (NMF), a special but important class of the general NMF, is demonstrated to be useful for data analysis and in particular for various clu…