540 citations · 1.1k across the 26 of their papers we have counts for
26 papers
Solving a class of multi-scale elliptic PDEs by means of Fourier-based mixed physics informed neural networks
Xi'an Li, Jinran Wu, You-Gan Wang +2
Deep neural networks have garnered widespread attention due to their simplicity and flexibility in the fields of engineering and scientific calculation. In this study, we probe int…
Understanding the Initial Condensation of Convolutional Neural Networks
Zhangchen Zhou, Hanxu Zhou, Yuqing Li +1
Previous research has shown that fully-connected networks with small initialization and gradient-based training methods exhibit a phenomenon known as condensation during training.…
Laplace-fPINNs: Laplace-based fractional physics-informed neural networks for solving forward and inverse problems of subdiffusion
Xiong-Bin Yan, Zhi-Qin John Xu, Zheng Ma
The use of Physics-informed neural networks (PINNs) has shown promise in solving forward and inverse problems of fractional diffusion equations. However, due to the fact that autom…
Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width
Hanxu Zhou, Qixuan Zhou, Zhenyuan Jin +3
Substantial work indicates that the dynamics of neural networks (NNs) is closely related to their initialization of parameters. Inspired by the phase diagram for two-layer ReLU NNs…
A multi-scale sampling method for accurate and robust deep neural network to predict combustion chemical kinetics
Tianhan Zhang, Yuxiao Yi, Yifan Xu +4
Machine learning has long been considered as a black box for predicting combustion chemical kinetics due to the extremely large number of parameters and the lack of evaluation stan…
A deep learning-based model reduction (DeePMR) method for simplifying chemical kinetics
Zhiwei Wang, Yaoyu Zhang, Enhan Zhao +4
A deep learning-based model reduction (DeePMR) method for simplifying chemical kinetics is proposed and validated using high-temperature auto-ignitions, perfectly stirred reactors…
Force-in-domain GAN inversion
Guangjie Leng, Yekun Zhu, Zhi-Qin John Xu
Empirical works suggest that various semantics emerge in the latent space of Generative Adversarial Networks (GANs) when being trained to generate images. To perform real image edi…
Linear Frequency Principle Model to Understand the Absence of Overfitting in Neural Networks
Yaoyu Zhang, Tao Luo, Zheng Ma +1
Why heavily parameterized neural networks (NNs) do not overfit the data is an important long standing open question. We propose a phenomenological model of the NN training to expla…
Frequency Principle in Deep Learning Beyond Gradient-descent-based Training
Yuheng Ma, Zhi-Qin John Xu, Jiwei Zhang
Frequency perspective recently makes progress in understanding deep learning. It has been widely verified in both empirical and theoretical studies that deep neural networks (DNNs)…
Fourier-domain Variational Formulation and Its Well-posedness for Supervised Learning
Tao Luo, Zheng Ma, Zhiwei Wang +2
A supervised learning problem is to find a function in a hypothesis function space given values on isolated data points. Inspired by the frequency principle in neural networks, we…
A multi-scale DNN algorithm for nonlinear elliptic equations with multiple scales
Xi-An Li, Zhi-Qin John Xu, Lei Zhang
Algorithms based on deep neural networks (DNNs) have attracted increasing attention from the scientific computing community. DNN based algorithms are easy to implement, natural for…
A regularized deep matrix factorized model of matrix completion for image restoration
Zhemin Li, Zhi-Qin John Xu, Tao Luo +1
It has been an important approach of using matrix completion to perform image restoration. Most previous works on matrix completion focus on the low-rank property by imposing expli…
Deep frequency principle towards understanding why deeper learning is faster
Zhi-Qin John Xu, Hanxu Zhou
Understanding the effect of depth in deep learning is a critical problem. In this work, we utilize the Fourier analysis to empirically provide a promising mechanism to understand w…
Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains
Ziqi Liu, Wei Cai, Zhi-Qin John Xu
In this paper, we propose multi-scale deep neural networks (MscaleDNNs) using the idea of radial scaling in frequency domain and activation functions with compact support. The radi…
Phase diagram for two-layer ReLU neural networks at infinite-width limit
Tao Luo, Zhi-Qin John Xu, Zheng Ma +1
How neural network behaves during the training over different choices of hyperparameters is an important question in the study of neural networks. In this work, inspired by the pha…
Implicit bias with Ritz-Galerkin method in understanding deep learning for solving PDEs
Jihong Wang, Zhi-Qin John Xu, Jiwei Zhang +1
This paper aims at studying the difference between Ritz-Galerkin (R-G) method and deep neural network (DNN) method in solving partial differential equations (PDEs) to better unders…
A priori generalization error for two-layer ReLU neural network through minimum norm solution
Zhi-Qin John Xu, Jiwei Zhang, Yaoyu Zhang +1
We focus on estimating \emph{a priori} generalization error of two-layer ReLU neural networks (NNs) trained by mean squared error, which only depends on initial parameters and the…
Multi-scale Deep Neural Networks for Solving High Dimensional PDEs
Wei Cai, Zhi-Qin John Xu
In this paper, we propose the idea of radial scaling in frequency domain and activation functions with compact support to produce a multi-scale DNN (MscaleDNN), which will have the…
Theory of the Frequency Principle for General Deep Neural Networks
Tao Luo, Zheng Ma, Zhi-Qin John Xu +1
Along with fruitful applications of Deep Neural Networks (DNNs) to realistic problems, recently, some empirical studies of DNNs reported a universal phenomenon of Frequency Princip…
Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks
Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo +1
It remains a puzzle that why deep neural networks (DNNs), with more parameters than samples, often generalize well. An attempt of understanding this puzzle is to discover implicit…
A type of generalization error induced by initialization in deep neural networks
Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo +1
How initialization and loss function affect the learning of a deep neural network (DNN), specifically its generalization error, is an important problem in practice. In this work, b…
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo +2
We study the training process of Deep Neural Networks (DNNs) from the Fourier analysis perspective. We demonstrate a very universal Frequency Principle (F-Principle) -- DNNs often…
Maximum Entropy Principle Analysis in Network Systems with Short-time Recordings
Zhi-Qin John Xu, Jennifer Crodelle, Douglas Zhou +1
In many realistic systems, maximum entropy principle (MEP) analysis provides an effective characterization of the probability distribution of network states. However, to implement…
Dynamical and Coupling Structure of Pulse-Coupled Networks in Maximum Entropy Analysis
Zhi-Qin John Xu, Douglas Zhou, David Cai
Maximum entropy principle (MEP) analysis with few non-zero effective interactions successfully characterizes the distribution of dynamical states of pulse-coupled networks in many…