Continual Learning on Dynamic Graphs via Parameter Isolation

Abstract

Many real-world graph learning tasks require handling dynamic graphs where new nodes and edges emerge. Dynamic graph learning methods commonly suffer from the catastrophic forgetting problem, where knowledge learned for previous graphs is overwritten by updates for new graphs. To alleviate the problem, continual graph learning methods are proposed. However, existing continual graph learning methods aim to learn new patterns and maintain old ones with the same set of parameters of fixed size, and thus face a fundamental tradeoff between both goals. In this paper, we propose Parameter Isolation GNN (PI-GNN) for continual learning on dynamic graphs that circumvents the tradeoff via parameter isolation and expansion. Our motivation lies in that different parameters contribute to learning different graph patterns. Based on the idea, we expand model parameters to continually learn emerging graph patterns. Meanwhile, to effectively preserve knowledge for unaffected patterns, we find parameters that correspond to them via optimization and freeze them to prevent them from being rewritten. Experiments on eight real-world datasets corroborate the effectiveness of PI-GNN compared to state-of-the-art baselines.

References in corpus (12)

• DeepWalk: Online Learning of Social Representations
• Overcoming catastrophic forgetting in neural networks
• LINE: Large-scale Information Network Embedding
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks
• Node Embedding over Temporal Graphs
• Efficiently Leveraging Multi-level User Intent for Session-based Recommendation via Atten-Mixer Network
• Streaming Graph Neural Networks via Continual Learning
• Learning Multi-granularity User Intent Unit for Session-based Recommendation
• Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks
• Evolutionary Preference Learning via Graph Nested GRU ODE for Session-based Recommendation
• Meta-Weight Graph Neural Network: Push the Limits Beyond Global Homophily

Cited by in corpus (2)

• Predicting Information Pathways Across Online Communities
• Efficient and Robust Continual Graph Learning for Graph Classification in Biology