ACE: Adapting to Changing Environments for Semantic Segmentation

Abstract

Deep neural networks exhibit exceptional accuracy when they are trained and tested on the same data distributions. However, neural classifiers are often extremely brittle when confronted with domain shift---changes in the input distribution that occur over time. We present ACE, a framework for semantic segmentation that dynamically adapts to changing environments over the time. By aligning the distribution of labeled training data from the original source domain with the distribution of incoming data in a shifted domain, ACE synthesizes labeled training data for environments as it sees them. This stylized data is then used to update a segmentation model so that it performs well in new environments. To avoid forgetting knowledge from past environments, we introduce a memory that stores feature statistics from previously seen domains. These statistics can be used to replay images in any of the previously observed domains, thus preventing catastrophic forgetting. In addition to standard batch training using stochastic gradient decent (SGD), we also experiment with fast adaptation methods based on adaptive meta-learning. Extensive experiments are conducted on two datasets from SYNTHIA, the results demonstrate the effectiveness of the proposed approach when adapting to a number of tasks.

References in corpus (31)

• Distilling the Knowledge in a Neural Network
• Overcoming catastrophic forgetting in neural networks
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Unsupervised Domain Adaptation by Backpropagation
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Continual Learning Through Synaptic Intelligence
• Matching Networks for One Shot Learning
• Fully Convolutional Networks for Semantic Segmentation
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
• Invertible Conditional GANs for image editing
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Gradient Episodic Memory for Continual Learning
• Adversarial Discriminative Domain Adaptation
• Deep CORAL: Correlation Alignment for Deep Domain Adaptation
• Pyramid Scene Parsing Network
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• Multimodal Unsupervised Image-to-Image Translation
• Continual Learning with Deep Generative Replay
• Reinforced Continual Learning
• Playing for Data: Ground Truth from Computer Games
• Demystifying Neural Style Transfer
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• Fully Convolutional Adaptation Networks for Semantic Segmentation
• End-to-End Incremental Learning
• Effective Use of Synthetic Data for Urban Scene Semantic Segmentation
• Learning Residual Images for Face Attribute Manipulation

Cited by in corpus (4)

• Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment
• Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation
• Open-sourced Dataset Protection via Backdoor Watermarking
• Curriculum Manager for Source Selection in Multi-Source Domain Adaptation