540 citations · 833 across the 33 of their papers we have counts for
7 papers · 1 filter
Embedding Principle: a hierarchical structure of loss landscape of deep neural networks
Yaoyu Zhang, Yuqing Li, Zhongwang Zhang +2
We prove a general Embedding Principle of loss landscape of deep neural networks (NNs) that unravels a hierarchical structure of the loss landscape of NNs, i.e., loss landscape of…
Data-informed Deep Optimization
Lulu Zhang, Zhi-Qin John Xu, Yaoyu Zhang
Complex design problems are common in the scientific and industrial fields. In practice, objective functions or constraints of these problems often do not have explicit formulas, a…
MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs
Lulu Zhang, Tao Luo, Yaoyu Zhang +3
In this paper, we propose a a machine learning approach via model-operator-data network (MOD-Net) for solving PDEs. A MOD-Net is driven by a model to solve PDEs based on operator r…
Embedding Principle of Loss Landscape of Deep Neural Networks
Yaoyu Zhang, Zhongwang Zhang, Tao Luo +1
Understanding the structure of loss landscape of deep neural networks (DNNs)is obviously important. In this work, we prove an embedding principle that the loss landscape of a DNN "…
Towards Understanding the Condensation of Neural Networks at Initial Training
Hanxu Zhou, Qixuan Zhou, Tao Luo +2
Empirical works show that for ReLU neural networks (NNs) with small initialization, input weights of hidden neurons (the input weight of a hidden neuron consists of the weight from…
An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Network
Tao Luo, Zheng Ma, Zhiwei Wang +2
Deep neural network (DNN) usually learns the target function from low to high frequency, which is called frequency principle or spectral bias. This frequency principle sheds light…
Linear Frequency Principle Model to Understand the Absence of Overfitting in Neural Networks
Yaoyu Zhang, Tao Luo, Zheng Ma +1
Why heavily parameterized neural networks (NNs) do not overfit the data is an important long standing open question. We propose a phenomenological model of the NN training to expla…