Understanding Graph Neural Networks from Graph Signal Denoising Perspectives

Abstract

Graph neural networks (GNNs) have attracted much attention because of their excellent performance on tasks such as node classification. However, there is inadequate understanding on how and why GNNs work, especially for node representation learning. This paper aims to provide a theoretical framework to understand GNNs, specifically, spectral graph convolutional networks and graph attention networks, from graph signal denoising perspectives. Our framework shows that GNNs are implicitly solving graph signal denoising problems: spectral graph convolutions work as denoising node features, while graph attentions work as denoising edge weights. We also show that a linear self-attention mechanism is able to compete with the state-of-the-art graph attention methods. Our theoretical results further lead to two new models, GSDN-F and GSDN-EF, which work effectively for graphs with noisy node features and/or noisy edges. We validate our theoretical findings and also the effectiveness of our new models by experiments on benchmark datasets. The source code is available at \url{https://github.com/fuguoji/GSDN}.

arXiv comment19 pages, 8 figures

References in corpus (10)

• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Discrete Signal Processing on Graphs
• Deep Convolutional Networks on Graph-Structured Data
• Fast Graph Representation Learning with PyTorch Geometric
• Simplifying Graph Convolutional Networks
• Predicting multicellular function through multi-layer tissue networks
• Signal Recovery on Graphs: Variation Minimization
• Attention-based Graph Neural Network for Semi-supervised Learning
• Revisiting Graph Neural Networks: All We Have is Low-Pass Filters

Cited by in corpus (5)

• Graph signal processing for machine learning: A review and new perspectives
• Graph Signal Processing for Geometric Data and Beyond: Theory and Applications
• $p$-Laplacian Based Graph Neural Networks
• Large Graph Signal Denoising with Application to Differential Privacy
• Graph Denoising with Framelet Regularizer