Learning at a Glance: Towards Interpretable Data-limited Continual Semantic Segmentation via Semantic-Invariance Modelling
arXiv:2407.15429 · doi:10.1109/TPAMI.2024.3396809
Abstract
Continual semantic segmentation (CSS) based on incremental learning (IL) is a great endeavour in developing human-like segmentation models. However, current CSS approaches encounter challenges in the trade-off between preserving old knowledge and learning new ones, where they still need large-scale annotated data for incremental training and lack interpretability. In this paper, we present Learning at a Glance (LAG), an efficient, robust, human-like and interpretable approach for CSS. Specifically, LAG is a simple and model-agnostic architecture, yet it achieves competitive CSS efficiency with limited incremental data. Inspired by human-like recognition patterns, we propose a semantic-invariance modelling approach via semantic features decoupling that simultaneously reconciles solid knowledge inheritance and new-term learning. Concretely, the proposed decoupling manner includes two ways, i.e., channel-wise decoupling and spatial-level neuron-relevant semantic consistency. Our approach preserves semantic-invariant knowledge as solid prototypes to alleviate catastrophic forgetting, while also constraining sample-specific contents through an asymmetric contrastive learning method to enhance model robustness during IL steps. Experimental results in multiple datasets validate the effectiveness of the proposed method. Furthermore, we introduce a novel CSS protocol that better reflects realistic data-limited CSS settings, and LAG achieves superior performance under multiple data-limited conditions.
References in corpus (19)
- Distilling the Knowledge in a Neural Network
- Overcoming catastrophic forgetting in neural networks
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
- A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
- Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
- Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks
- A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
- Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
- Continual Lifelong Learning in Natural Language Processing: A Survey
- Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation
- SATS: Self-Attention Transfer for Continual Semantic Segmentation
- Prototype-Guided Memory Replay for Continual Learning
- A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application
- Inherit with Distillation and Evolve with Contrast: Exploring Class Incremental Semantic Segmentation Without Exemplar Memory
- Embedded Self-Distillation in Compact Multi-Branch Ensemble Network for Remote Sensing Scene Classification
- Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation