AKE-GNN: Effective Graph Learning with Adaptive Knowledge Exchange

Abstract

Graph Neural Networks (GNNs) have already been widely used in various graph mining tasks. However, recent works reveal that the learned weights (channels) in well-trained GNNs are highly redundant, which inevitably limits the performance of GNNs. Instead of removing these redundant channels for efficiency consideration, we aim to reactivate them to enlarge the representation capacity of GNNs for effective graph learning. In this paper, we propose to substitute these redundant channels with other informative channels to achieve this goal. We introduce a novel GNN learning framework named AKE-GNN, which performs the Adaptive Knowledge Exchange strategy among multiple graph views generated by graph augmentations. AKE-GNN first trains multiple GNNs each corresponding to one graph view to obtain informative channels. Then, AKE-GNN iteratively exchanges redundant channels in the weight parameter matrix of one GNN with informative channels of another GNN in a layer-wise manner. Additionally, existing GNNs can be seamlessly incorporated into our framework. AKE-GNN achieves superior performance compared with various baselines across a suite of experiments on node classification, link prediction, and graph classification. In particular, we conduct a series of experiments on 15 public benchmark datasets, 8 popular GNN models, and 3 graph tasks and show that AKE-GNN consistently outperforms existing popular GNN models and even their ensembles. Extensive ablation studies and analyses on knowledge exchange methods validate the effectiveness of AKE-GNN.

References in corpus (14)

• Distilling the Knowledge in a Neural Network
• Simplifying Graph Convolutional Networks
• Representation Learning on Graphs with Jumping Knowledge Networks
• Opening the Black Box of Deep Neural Networks via Information
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Towards Deeper Graph Neural Networks
• Simple and Deep Graph Convolutional Networks
• Contrastive Multi-View Representation Learning on Graphs
• PairNorm: Tackling Oversmoothing in GNNs
• Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction
• Geom-GCN: Geometric Graph Convolutional Networks
• A Fair Comparison of Graph Neural Networks for Graph Classification
• Multi-View Graph Neural Networks for Molecular Property Prediction
• Calibrating and Improving Graph Contrastive Learning