Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies

Abstract

Intelligent behaviour in the real-world requires the ability to acquire new knowledge from an ongoing sequence of experiences while preserving and reusing past knowledge. We propose a novel algorithm for unsupervised representation learning from piece-wise stationary visual data: Variational Autoencoder with Shared Embeddings (VASE). Based on the Minimum Description Length principle, VASE automatically detects shifts in the data distribution and allocates spare representational capacity to new knowledge, while simultaneously protecting previously learnt representations from catastrophic forgetting. Our approach encourages the learnt representations to be disentangled, which imparts a number of desirable properties: VASE can deal sensibly with ambiguous inputs, it can enhance its own representations through imagination-based exploration, and most importantly, it exhibits semantically meaningful sharing of latents between different datasets. Compared to baselines with entangled representations, our approach is able to reason beyond surface-level statistics and perform semantically meaningful cross-domain inference.

Cited by in corpus (24)

• Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons
• Are Disentangled Representations Helpful for Abstract Visual Reasoning?
• Lifelong Generative Modeling
• Lifelong Teacher-Student Network Learning
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• Unsupervised Model Selection for Variational Disentangled Representation Learning
• DisCoRL: Continual Reinforcement Learning via Policy Distillation
• Multi-Task Incremental Learning for Object Detection
• Toward Understanding Catastrophic Forgetting in Continual Learning
• Continual State Representation Learning for Reinforcement Learning using Generative Replay
• Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
• Lifelong Learning of Compositional Structures
• Learning latent representations across multiple data domains using Lifelong VAEGAN
• Disentangling by Subspace Diffusion
• Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer
• Task-agnostic Continual Learning with Hybrid Probabilistic Models
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
• Graph-Based Continual Learning
• Odd-One-Out Representation Learning
• A Procedural World Generation Framework for Systematic Evaluation of Continual Learning
• Growing Representation Learning
• Boxhead: A Dataset for Learning Hierarchical Representations
• Lifelong Mixture of Variational Autoencoders
• Recoding latent sentence representations -- Dynamic gradient-based activation modification in RNNs