128 citations · 540 across the 44 of their papers we have counts for
46 papers
Second-Order KKT Guarantees for Bregman ADMM in Nonconvex and Non-Lipschitz Optimization
Shuang Li, Zhihui Zhu, Qiuwei Li
We analyze Bregman ADMM for nonconvex linearly constrained problems under two-sided relative smoothness, a condition that replaces the standard Lipschitz gradient assumption with a…
A Provable Splitting Approach for Symmetric Nonnegative Matrix Factorization
Xiao Li, Zhihui Zhu, Qiuwei Li +1
The symmetric Nonnegative Matrix Factorization (NMF), a special but important class of the general NMF, has found numerous applications in data analysis such as various clustering…
Are All Losses Created Equal: A Neural Collapse Perspective
Jinxin Zhou, Chong You, Xiao Li +4
While cross entropy (CE) is the most commonly used loss to train deep neural networks for classification tasks, many alternative losses have been developed to obtain better empiric…
Sparsity-guided Network Design for Frame Interpolation
Tianyu Ding, Luming Liang, Zhihui Zhu +2
DNN-based frame interpolation, which generates intermediate frames from two consecutive frames, is often dependent on model architectures with a large number of features, preventin…
A Validation Approach to Over-parameterized Matrix and Image Recovery
Lijun Ding, Zhen Qin, Liwei Jiang +2
This paper studies the problem of recovering a low-rank matrix from several noisy random linear measurements. We consider the setting where the rank of the ground-truth matrix is u…
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold
Can Yaras, Peng Wang, Zhihui Zhu +2
When training overparameterized deep networks for classification tasks, it has been widely observed that the learned features exhibit a so-called "neural collapse" phenomenon. More…
Quantum state tomography with tensor train cross approximation
Alexander Lidiak, Casey Jameson, Zhen Qin +4
It has been recently shown that a state generated by a one-dimensional noisy quantum computer is well approximated by a matrix product operator with a finite bond dimension indepen…
Error Analysis of Tensor-Train Cross Approximation
Zhen Qin, Alexander Lidiak, Zhexuan Gong +3
Tensor train decomposition is widely used in machine learning and quantum physics due to its concise representation of high-dimensional tensors, overcoming the curse of dimensional…
On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features
Jinxin Zhou, Xiao Li, Tianyu Ding +3
When training deep neural networks for classification tasks, an intriguing empirical phenomenon has been widely observed in the last-layer classifiers and features, where (i) the c…
Robust Training under Label Noise by Over-parameterization
Sheng Liu, Zhihui Zhu, Qing Qu +1
Recently, over-parameterized deep networks, with increasingly more network parameters than training samples, have dominated the performances of modern machine learning. However, wh…
Rank Overspecified Robust Matrix Recovery: Subgradient Method and Exact Recovery
Lijun Ding, Liwei Jiang, Yudong Chen +2
We study the robust recovery of a low-rank matrix from sparsely and grossly corrupted Gaussian measurements, with no prior knowledge on the intrinsic rank. We consider the robust m…
Only Train Once: A One-Shot Neural Network Training And Pruning Framework
Tianyi Chen, Bo Ji, Tianyu Ding +7
Structured pruning is a commonly used technique in deploying deep neural networks (DNNs) onto resource-constrained devices. However, the existing pruning methods are usually heuris…
A Geometric Analysis of Neural Collapse with Unconstrained Features
Zhihui Zhu, Tianyu Ding, Jinxin Zhou +4
We provide the first global optimization landscape analysis of -- an intriguing empirical phenomenon that arises in the last-layer classifiers and features of ne…
CDFI: Compression-Driven Network Design for Frame Interpolation
Tianyu Ding, Luming Liang, Zhihui Zhu +1
DNN-based frame interpolation--that generates the intermediate frames given two consecutive frames--typically relies on heavy model architectures with a huge number of features, pr…
Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training
Sheng Liu, Xiao Li, Yuexiang Zhai +4
Normalization techniques have become a basic component in modern convolutional neural networks (ConvNets). In particular, many recent works demonstrate that promoting the orthogona…
Half-Space Proximal Stochastic Gradient Method for Group-Sparsity Regularized Problem
Tianyi Chen, Guanyi Wang, Tianyu Ding +3
Optimizing with group sparsity is significant in enhancing model interpretability in machining learning applications, e.g., feature selection, compressed sensing and model compress…
Robust Recovery via Implicit Bias of Discrepant Learning Rates for Double Over-parameterization
Chong You, Zhihui Zhu, Qing Qu +1
Recent advances have shown that implicit bias of gradient descent on over-parameterized models enables the recovery of low-rank matrices from linear measurements, even with no prio…
Recovery and Generalization in Over-Realized Dictionary Learning
Jeremias Sulam, Chong You, Zhihui Zhu
In over two decades of research, the field of dictionary learning has gathered a large collection of successful applications, and theoretical guarantees for model recovery are know…
Orthant Based Proximal Stochastic Gradient Method for -Regularized Optimization
Tianyi Chen, Tianyu Ding, Bo Ji +6
Sparsity-inducing regularization problems are ubiquitous in machine learning applications, ranging from feature selection to model compression. In this paper, we present a novel st…
The Global Geometry of Centralized and Distributed Low-rank Matrix Recovery without Regularization
Shuang Li, Qiuwei Li, Zhihui Zhu +2
Low-rank matrix recovery is a fundamental problem in signal processing and machine learning. A recent very popular approach to recovering a low-rank matrix X is to factorize it as…
Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications
Qing Qu, Zhihui Zhu, Xiao Li +3
The problem of finding the sparsest vector (direction) in a low dimensional subspace can be considered as a homogeneous variant of the sparse recovery problem, which finds applicat…
Analysis of the Optimization Landscapes for Overcomplete Representation Learning
Qing Qu, Yuexiang Zhai, Xiao Li +2
We study nonconvex optimization landscapes for learning overcomplete representations, including learning (i) sparsely used overcomplete dictionaries and (ii) convolutional dictiona…
Viewpoint-Aware Loss with Angular Regularization for Person Re-Identification
Zhihui Zhu, Xinyang Jiang, Feng Zheng +4
Although great progress in supervised person re-identification (Re-ID) has been made recently, due to the viewpoint variation of a person, Re-ID remains a massive visual challenge.…
Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods
Xiao Li, Shixiang Chen, Zengde Deng +3
We consider a class of nonsmooth optimization problems over the Stiefel manifold, in which the objective function is weakly convex in the ambient Euclidean space. Such problems are…