Multi-View Class Incremental Learning

Abstract

Multi-view learning (MVL) has gained great success in integrating information from multiple perspectives of a dataset to improve downstream task performance. To make MVL methods more practical in an open-ended environment, this paper investigates a novel paradigm called multi-view class incremental learning (MVCIL), where a single model incrementally classifies new classes from a continual stream of views, requiring no access to earlier views of data. However, MVCIL is challenged by the catastrophic forgetting of old information and the interference with learning new concepts. To address this, we first develop a randomization-based representation learning technique serving for feature extraction to guarantee their separate view-optimal working states, during which multiple views belonging to a class are presented sequentially; Then, we integrate them one by one in the orthogonality fusion subspace spanned by the extracted features; Finally, we introduce selective weight consolidation for learning-without-forgetting decision-making while encountering new classes. Extensive experiments on synthetic and real-world datasets validate the effectiveness of our approach.

arXiv commentAccepted to Information Fusion

References in corpus (15)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Overcoming catastrophic forgetting in neural networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Learning deep representations by mutual information estimation and maximization
• A continual learning survey: Defying forgetting in classification tasks
• Continual Learning of Context-dependent Processing in Neural Networks
• Overcoming catastrophic forgetting with hard attention to the task
• Generalized Multi-view Embedding for Visual Recognition and Cross-modal Retrieval
• Learning Robust Representations via Multi-View Information Bottleneck
• Trusted Multi-View Classification
• Domain-incremental Cardiac Image Segmentation with Style-oriented Replay and Domain-sensitive Feature Whitening
• Streaming View Learning
• Continual Learning Using Multi-view Task Conditional Neural Networks
• IF2Net: Innately Forgetting-Free Networks for Continual Learning
• Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning