43 citations · 177 across the 17 of their papers we have counts for
18 papers
Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modeling
Xiao Li, Zekai Zhang, Xiang Li +4
Diffusion models, though originally designed for generative tasks, have demonstrated impressive self-supervised representation learning capabilities. A particularly intriguing phen…
Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination
Peng Wang, Xiao Li, Can Yaras +4
Over the past decade, deep learning has proven to be a highly effective tool for learning meaningful features from raw data. However, it remains an open question how deep networks…
ReSync: Riemannian Subgradient-based Robust Rotation Synchronization
Huikang Liu, Xiao Li, Anthony Man-Cho So
This work presents ReSync, a Riemannian subgradient-based algorithm for solving the robust rotation synchronization problem, which arises in various engineering applications. ReSyn…
A Provable Splitting Approach for Symmetric Nonnegative Matrix Factorization
Xiao Li, Zhihui Zhu, Qiuwei Li +1
The symmetric Nonnegative Matrix Factorization (NMF), a special but important class of the general NMF, has found numerous applications in data analysis such as various clustering…
Understanding and Improving Transfer Learning of Deep Models via Neural Collapse
Xiao Li, Sheng Liu, Jinxin Zhou +4
With the ever-increasing complexity of large-scale pre-trained models coupled with a shortage of labeled data for downstream training, transfer learning has become the primary appr…
Are All Losses Created Equal: A Neural Collapse Perspective
Jinxin Zhou, Chong You, Xiao Li +4
While cross entropy (CE) is the most commonly used loss to train deep neural networks for classification tasks, many alternative losses have been developed to obtain better empiric…
On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features
Jinxin Zhou, Xiao Li, Tianyu Ding +3
When training deep neural networks for classification tasks, an intriguing empirical phenomenon has been widely observed in the last-layer classifiers and features, where (i) the c…
A Geometric Analysis of Neural Collapse with Unconstrained Features
Zhihui Zhu, Tianyu Ding, Jinxin Zhou +4
We provide the first global optimization landscape analysis of -- an intriguing empirical phenomenon that arises in the last-layer classifiers and features of ne…
Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training
Sheng Liu, Xiao Li, Yuexiang Zhai +4
Normalization techniques have become a basic component in modern convolutional neural networks (ConvNets). In particular, many recent works demonstrate that promoting the orthogona…
Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications
Qing Qu, Zhihui Zhu, Xiao Li +3
The problem of finding the sparsest vector (direction) in a low dimensional subspace can be considered as a homogeneous variant of the sparse recovery problem, which finds applicat…
Analysis of the Optimization Landscapes for Overcomplete Representation Learning
Qing Qu, Yuexiang Zhai, Xiao Li +2
We study nonconvex optimization landscapes for learning overcomplete representations, including learning (i) sparsely used overcomplete dictionaries and (ii) convolutional dictiona…
Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods
Xiao Li, Shixiang Chen, Zengde Deng +3
We consider a class of nonsmooth optimization problems over the Stiefel manifold, in which the objective function is weakly convex in the ambient Euclidean space. Such problems are…
A Nonconvex Approach for Exact and Efficient Multichannel Sparse Blind Deconvolution
Qing Qu, Xiao Li, Zhihui Zhu
We study the multi-channel sparse blind deconvolution (MCS-BD) problem, whose task is to simultaneously recover a kernel and multiple sparse inputs $\{\mathbf x_i\}_{i=…
Incremental Methods for Weakly Convex Optimization
Xiao Li, Zhihui Zhu, Anthony Man-Cho So +1
Incremental methods are widely utilized for solving finite-sum optimization problems in machine learning and signal processing. In this paper, we study a family of incremental meth…
Dropping Symmetry for Fast Symmetric Nonnegative Matrix Factorization
Zhihui Zhu, Xiao Li, Kai Liu +1
Symmetric nonnegative matrix factorization (NMF), a special but important class of the general NMF, is demonstrated to be useful for data analysis and in particular for various clu…
Nonconvex Robust Low-rank Matrix Recovery
Xiao Li, Zhihui Zhu, Anthony Man-Cho So +1
In this paper we study the problem of recovering a low-rank matrix from a number of random linear measurements that are corrupted by outliers taking arbitrary values. We consider a…
Convergence Analysis of Alternating Projection Method for Nonconvex Sets
Zhihui Zhu, Xiao Li
Alternating projection method has been used in a wide range of engineering applications since it is a gradient-free method (without requiring tuning the step size) and usually has…
Optimized Structured Sparse Sensing Matrices for Compressive Sensing
Tao Hong, Xiao Li, Zhihui Zhu +1
We consider designing a robust structured sparse sensing matrix consisting of a sparse matrix with a few non-zero entries per row and a dense base matrix for capturing signals effi…