How Framelets Enhance Graph Neural Networks

Abstract

This paper presents a new approach for assembling graph neural networks based on framelet transforms. The latter provides a multi-scale representation for graph-structured data. We decompose an input graph into low-pass and high-pass frequencies coefficients for network training, which then defines a framelet-based graph convolution. The framelet decomposition naturally induces a graph pooling strategy by aggregating the graph feature into low-pass and high-pass spectra, which considers both the feature values and geometry of the graph data and conserves the total information. The graph neural networks with the proposed framelet convolution and pooling achieve state-of-the-art performance in many node and graph prediction tasks. Moreover, we propose shrinkage as a new activation for the framelet convolution, which thresholds high-frequency information at different scales. Compared to ReLU, shrinkage activation improves model performance on denoising and signal compression: noises in both node and structure can be significantly reduced by accurately cutting off the high-pass coefficients from framelet decomposition, and the signal can be compressed to less than half its original size with well-preserved prediction performance.

arXiv comment24 pages, 17 figures, 8 tables, ICML2021 (fix typos)

References in corpus (15)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Neural Message Passing for Quantum Chemistry
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• Simple and Deep Graph Convolutional Networks
• TUDataset: A collection of benchmark datasets for learning with graphs
• DeeperGCN: All You Need to Train Deeper GCNs
• Towards Sparse Hierarchical Graph Classifiers
• SIGN: Scalable Inception Graph Neural Networks
• LanczosNet: Multi-Scale Deep Graph Convolutional Networks
• On the Spectral Bias of Neural Networks
• Combining Label Propagation and Simple Models Out-performs Graph Neural Networks
• Graph Wavelet Neural Network
• Path Integral Based Convolution and Pooling for Graph Neural Networks
• Decimated Framelet System on Graphs and Fast G-Framelet Transforms
• MathNet: Haar-Like Wavelet Multiresolution-Analysis for Graph Representation and Learning

Cited by in corpus (1)

• Spectral Transform Forms Scalable Transformer