540 citations · 833 across the 33 of their papers we have counts for
5 papers · 1 filter
A priori generalization error for two-layer ReLU neural network through minimum norm solution
Zhi-Qin John Xu, Jiwei Zhang, Yaoyu Zhang +1
We focus on estimating \emph{a priori} generalization error of two-layer ReLU neural networks (NNs) trained by mean squared error, which only depends on initial parameters and the…
Theory of the Frequency Principle for General Deep Neural Networks
Tao Luo, Zheng Ma, Zhi-Qin John Xu +1
Along with fruitful applications of Deep Neural Networks (DNNs) to realistic problems, recently, some empirical studies of DNNs reported a universal phenomenon of Frequency Princip…
Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks
Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo +1
It remains a puzzle that why deep neural networks (DNNs), with more parameters than samples, often generalize well. An attempt of understanding this puzzle is to discover implicit…
A type of generalization error induced by initialization in deep neural networks
Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo +1
How initialization and loss function affect the learning of a deep neural network (DNN), specifically its generalization error, is an important problem in practice. In this work, b…
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo +2
We study the training process of Deep Neural Networks (DNNs) from the Fourier analysis perspective. We demonstrate a very universal Frequency Principle (F-Principle) -- DNNs often…