1 citations · 1 across the 9 of their papers we have counts for
9 papers
Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting
Leyang Zhang, Yaoyu Zhang, Tao Luo
This paper investigates the sample dependence of critical points for neural networks. We introduce a sample-independent critical lifting operator that associates a parameter of one…
Linear Independence of Generalized Neurons and Related Functions
Leyang Zhang
The linear independence of neurons plays a significant role in theoretical analysis of neural networks. Specifically, given neurons $H_1, ..., H_n: \bR^N \times \bR^d \to \bR$, we…
Local Linear Recovery Guarantee of Deep Neural Networks at Overparameterization
Yaoyu Zhang, Leyang Zhang, Zhongwang Zhang +1
Determining whether deep neural network (DNN) models can reliably recover target functions at overparameterization is a critical yet complex issue in the theory of deep learning. T…
Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networks
Leyang Zhang, Yaoyu Zhang, Tao Luo
This paper presents a comprehensive analysis of critical point sets in two-layer neural networks. To study such complex entities, we introduce the critical embedding operator and c…
Gradient Flow in Morse and Analytic Settings
Leyang Zhang
This report is actually the in-class project of MATH489 FA2022, Dynamics and Differential Equations in University of Illinois Urbana-Champaign. The report is mainly about gradient…
Geometry and Local Recovery of Global Minima of Two-layer Neural Networks at Overparameterization
Leyang Zhang, Yaoyu Zhang, Tao Luo
Under mild assumptions, we investigate the geometry of the loss landscape for two-layer neural networks in the vicinity of global minima. Utilizing novel techniques, we demonstrate…
Optimistic Estimate Uncovers the Potential of Nonlinear Models
Yaoyu Zhang, Zhongwang Zhang, Leyang Zhang +3
We propose an optimistic estimate to evaluate the best possible fitting performance of nonlinear models. It yields an optimistic sample size that quantifies the smallest possible s…
Linear Stability Hypothesis and Rank Stratification for Nonlinear Models
Yaoyu Zhang, Zhongwang Zhang, Leyang Zhang +3
Models with nonlinear architectures/parameterizations such as deep neural networks (DNNs) are well known for their mysteriously good generalization performance at overparameterizat…
Limitation of Characterizing Implicit Regularization by Data-independent Functions
Leyang Zhang, Zhi-Qin John Xu, Tao Luo +1
In recent years, understanding the implicit regularization of neural networks (NNs) has become a central task in deep learning theory. However, implicit regularization is itself no…