Invariance-Preserving Localized Activation Functions for Graph Neural Networks

Abstract

Graph signals are signals with an irregular structure that can be described by a graph. Graph neural networks (GNNs) are information processing architectures tailored to these graph signals and made of stacked layers that compose graph convolutional filters with nonlinear activation functions. Graph convolutions endow GNNs with invariance to permutations of the graph nodes' labels. In this paper, we consider the design of trainable nonlinear activation functions that take into consideration the structure of the graph. This is accomplished by using graph median filters and graph max filters, which mimic linear graph convolutions and are shown to retain the permutation invariance of GNNs. We also discuss modifications to the backpropagation algorithm necessary to train local activation functions. The advantages of localized activation function architectures are demonstrated in four numerical experiments: source localization on synthetic graphs, authorship attribution of 19th century novels, movie recommender systems and scientific article classification. In all cases, localized activation functions are shown to improve model capacity.

arXiv commentAccepted at TSP

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• Graph Convolutional Neural Networks for Web-Scale Recommender Systems
• Discrete Signal Processing on Graphs
• Deep Convolutional Networks on Graph-Structured Data
• Convolutional Neural Network Architectures for Signals Supported on Graphs
• Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks
• Stability Properties of Graph Neural Networks
• Blind Identification of Graph Filters
• Authorship Attribution through Function Word Adjacency Networks
• Stability of Graph Scattering Transforms
• Diffusion Scattering Transforms on Graphs
• CNN as Guided Multi-layer RECOS Transform

Cited by in corpus (9)

• Stability Properties of Graph Neural Networks
• Gated Graph Recurrent Neural Networks
• Graphon Signal Processing
• Graphon Neural Networks and the Transferability of Graph Neural Networks
• Constrained Learning with Non-Convex Losses
• Signal Processing on Directed Graphs
• Probably Approximately Correct Constrained Learning
• Deep Constraint-based Propagation in Graph Neural Networks
• Graphs for deep learning representations