Graph Convolutional Neural Networks with Node Transition Probability-based Message Passing and DropNode Regularization

Abstract

Graph convolutional neural networks (GCNNs) have received much attention recently, owing to their capability in handling graph-structured data. Among the existing GCNNs, many methods can be viewed as instances of a neural message passing motif; features of nodes are passed around their neighbors, aggregated and transformed to produce better nodes' representations. Nevertheless, these methods seldom use node transition probabilities, a measure that has been found useful in exploring graphs. Furthermore, when the transition probabilities are used, their transition direction is often improperly considered in the feature aggregation step, resulting in an inefficient weighting scheme. In addition, although a great number of GCNN models with increasing level of complexity have been introduced, the GCNNs often suffer from over-fitting when being trained on small graphs. Another issue of the GCNNs is over-smoothing, which tends to make nodes' representations indistinguishable. This work presents a new method to improve the message passing process based on node transition probabilities by properly considering the transition direction, leading to a better weighting scheme in nodes' features aggregation compared to the existing counterpart. Moreover, we propose a novel regularization method termed DropNode to address the over-fitting and over-smoothing issues simultaneously. DropNode randomly discards part of a graph, thus it creates multiple deformed versions of the graph, leading to data augmentation regularization effect. Additionally, DropNode lessens the connectivity of the graph, mitigating the effect of over-smoothing in deep GCNNs. Extensive experiments on eight benchmark datasets for node and graph classification tasks demonstrate the effectiveness of the proposed methods in comparison with the state of the art.

arXiv commentExpert Systems with Applications, graph-based deep learning, graph neural networks, document classification

References in corpus (27)

• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Improved Regularization of Convolutional Neural Networks with Cutout
• Spectral Networks and Locally Connected Networks on Graphs
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Graph Neural Networks: A Review of Methods and Applications
• Deep Convolutional Networks on Graph-Structured Data
• Convolutional Networks on Graphs for Learning Molecular Fingerprints
• Hierarchical Graph Representation Learning with Differentiable Pooling
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Interaction Networks for Learning about Objects, Relations and Physics
• Diffusion-Convolutional Neural Networks
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
• Graph Neural Networks with convolutional ARMA filters
• Gated Graph Sequence Neural Networks
• Self-Attention Graph Pooling
• Fake News Detection on Social Media using Geometric Deep Learning
• GMNN: Graph Markov Neural Networks
• Deep Graph Infomax
• Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs
• Multiview Deep Learning for Predicting Twitter Users' Location
• On Graph Classification Networks, Datasets and Baselines
• Dynamics Based Features For Graph Classification
• Matrix Completion With Variational Graph Autoencoders: Application in Hyperlocal Air Quality Inference