540 citations · 841 across the 27 of their papers we have counts for
7 papers · 1 filter
Embedding Principle: a hierarchical structure of loss landscape of deep neural networks
Yaoyu Zhang, Yuqing Li, Zhongwang Zhang +2
We prove a general Embedding Principle of loss landscape of deep neural networks (NNs) that unravels a hierarchical structure of the loss landscape of NNs, i.e., loss landscape of…
MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs
Lulu Zhang, Tao Luo, Yaoyu Zhang +3
In this paper, we propose a a machine learning approach via model-operator-data network (MOD-Net) for solving PDEs. A MOD-Net is driven by a model to solve PDEs based on operator r…
Embedding Principle of Loss Landscape of Deep Neural Networks
Yaoyu Zhang, Zhongwang Zhang, Tao Luo +1
Understanding the structure of loss landscape of deep neural networks (DNNs)is obviously important. In this work, we prove an embedding principle that the loss landscape of a DNN "…
Towards Understanding the Condensation of Neural Networks at Initial Training
Hanxu Zhou, Qixuan Zhou, Tao Luo +2
Empirical works show that for ReLU neural networks (NNs) with small initialization, input weights of hidden neurons (the input weight of a hidden neuron consists of the weight from…
An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Network
Tao Luo, Zheng Ma, Zhiwei Wang +2
Deep neural network (DNN) usually learns the target function from low to high frequency, which is called frequency principle or spectral bias. This frequency principle sheds light…
Nonlinear Weighted Directed Acyclic Graph and A Priori Estimates for Neural Networks
Yuqing Li, Tao Luo, Chao Ma
In an attempt to better understand structural benefits and generalization power of deep neural networks, we firstly present a novel graph theoretical formulation of neural network…
Linear Frequency Principle Model to Understand the Absence of Overfitting in Neural Networks
Yaoyu Zhang, Tao Luo, Zheng Ma +1
Why heavily parameterized neural networks (NNs) do not overfit the data is an important long standing open question. We propose a phenomenological model of the NN training to expla…