Catastrophic Forgetting in Deep Graph Networks: an Introductory Benchmark for Graph Classification
arXiv:2103.11750
Abstract
In this work, we study the phenomenon of catastrophic forgetting in the graph representation learning scenario. The primary objective of the analysis is to understand whether classical continual learning techniques for flat and sequential data have a tangible impact on performances when applied to graph data. To do so, we experiment with a structure-agnostic model and a deep graph network in a robust and controlled environment on three different datasets. The benchmark is complemented by an investigation on the effect of structure-preserving regularization techniques on catastrophic forgetting. We find that replay is the most effective strategy in so far, which also benefits the most from the use of regularization. Our findings suggest interesting future research at the intersection of the continual and graph representation learning fields. Finally, we provide researchers with a flexible software framework to reproduce our results and carry out further experiments.
arXiv commentAccepted at the 2021 Web Conference Workshop on Graph Learning Benchmarks (GLB 2021). Code available at https://github.com/diningphil/continual_learning_for_graphs
References in corpus (9)
- Semi-Supervised Classification with Graph Convolutional Networks
- Geometric deep learning: going beyond Euclidean data
- Fast Graph Representation Learning with PyTorch Geometric
- Efficient Lifelong Learning with A-GEM
- On Tiny Episodic Memories in Continual Learning
- A Fair Comparison of Graph Neural Networks for Graph Classification
- Continual Learning with Gated Incremental Memories for sequential data processing
- Regularization Shortcomings for Continual Learning
- Continual Learning of New Sound Classes using Generative Replay