540 citations · 908 across the 23 of their papers we have counts for
14 papers · 1 filter
GEDI: A Graph-based End-to-end Data Imputation Framework
Katrina Chen, Xiuqin Liang, Zheng Ma +1
Data imputation is an effective way to handle missing data, which is common in practical applications. In this study, we propose and test a novel data imputation process that achie…
An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Network
Tao Luo, Zheng Ma, Zhiwei Wang +2
Deep neural network (DNN) usually learns the target function from low to high frequency, which is called frequency principle or spectral bias. This frequency principle sheds light…
Linear Frequency Principle Model to Understand the Absence of Overfitting in Neural Networks
Yaoyu Zhang, Tao Luo, Zheng Ma +1
Why heavily parameterized neural networks (NNs) do not overfit the data is an important long standing open question. We propose a phenomenological model of the NN training to expla…
On the exact computation of linear frequency principle dynamics and its generalization
Tao Luo, Zheng Ma, Zhi-Qin John Xu +1
Recent works show an intriguing phenomenon of Frequency Principle (F-Principle) that deep neural networks (DNNs) fit the target function from low to high frequency during the train…
Path Integral Based Convolution and Pooling for Graph Neural Networks
Zheng Ma, Junyu Xuan, Yu Guang Wang +2
Graph neural networks (GNNs) extends the functionality of traditional neural networks to graph-structured data. Similar to CNNs, an optimized design of graph convolution and poolin…
Phase diagram for two-layer ReLU neural networks at infinite-width limit
Tao Luo, Zhi-Qin John Xu, Zheng Ma +1
How neural network behaves during the training over different choices of hyperparameters is an important question in the study of neural networks. In this work, inspired by the pha…
Haar Graph Pooling
Yu Guang Wang, Ming Li, Zheng Ma +3
Deep Graph Neural Networks (GNNs) are useful models for graph classification and graph-based regression tasks. In these tasks, graph pooling is a critical ingredient by which GNNs…
Fast Haar Transforms for Graph Neural Networks
Ming Li, Zheng Ma, Yu Guang Wang +1
Graph Neural Networks (GNNs) have become a topic of intense research recently due to their powerful capability in high-dimensional classification and regression tasks for graph-str…
Theory of the Frequency Principle for General Deep Neural Networks
Tao Luo, Zheng Ma, Zhi-Qin John Xu +1
Along with fruitful applications of Deep Neural Networks (DNNs) to realistic problems, recently, some empirical studies of DNNs reported a universal phenomenon of Frequency Princip…
Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks
Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo +1
It remains a puzzle that why deep neural networks (DNNs), with more parameters than samples, often generalize well. An attempt of understanding this puzzle is to discover implicit…
A type of generalization error induced by initialization in deep neural networks
Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo +1
How initialization and loss function affect the learning of a deep neural network (DNN), specifically its generalization error, is an important problem in practice. In this work, b…
PAN: Path Integral Based Convolution for Deep Graph Neural Networks
Zheng Ma, Ming Li, Yuguang Wang
Convolution operations designed for graph-structured data usually utilize the graph Laplacian, which can be seen as message passing between the adjacent neighbors through a generic…
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo +2
We study the training process of Deep Neural Networks (DNNs) from the Fourier analysis perspective. We demonstrate a very universal Frequency Principle (F-Principle) -- DNNs often…
NADPEx: An on-policy temporally consistent exploration method for deep reinforcement learning
Sirui Xie, Junning Huang, Lanxin Lei +4
Reinforcement learning agents need exploratory behaviors to escape from local optima. These behaviors may include both immediate dithering perturbation and temporally consistent ex…