Robust Unsupervised Domain Adaptation for Neural Networks via Moment Alignment

Abstract

A novel approach for unsupervised domain adaptation for neural networks is proposed. It relies on metric-based regularization of the learning process. The metric-based regularization aims at domain-invariant latent feature representations by means of maximizing the similarity between domain-specific activation distributions. The proposed metric results from modifying an integral probability metric such that it becomes less translation-sensitive on a polynomial function space. The metric has an intuitive interpretation in the dual space as the sum of differences of higher order central moments of the corresponding activation distributions. Under appropriate assumptions on the input distributions, error minimization is proven for the continuous case. As demonstrated by an analysis of standard benchmark experiments for sentiment analysis, object recognition and digit recognition, the outlined approach is robust regarding parameter changes and achieves higher classification accuracies than comparable approaches. The source code is available at https://github.com/wzell/mann.

arXiv commentPreliminary version of this work appeared in ICLR

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• ADADELTA: An Adaptive Learning Rate Method
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Domain Separation Networks
• Marginalized Denoising Autoencoders for Domain Adaptation
• Domain Adaptation: Learning Bounds and Algorithms
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Revisiting Batch Normalization For Practical Domain Adaptation
• The Variational Fair Autoencoder

Cited by in corpus (14)

• A review of domain adaptation without target labels
• Semi-supervised representation learning via dual autoencoders for domain adaptation
• Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data
• Collaborate to Adapt: Source-Free Graph Domain Adaptation via Bi-directional Adaptation
• Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance
• Distribution Consistency based Self-Training for Graph Neural Networks with Sparse Labels
• A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasets
• On generalization in moment-based domain adaptation
• CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs
• Moment-Based Domain Adaptation: Learning Bounds and Algorithms
• Graceful Degradation and Related Fields
• Improved Hard Example Mining by Discovering Attribute-based Hard Person Identity
• HC-GST: Heterophily-aware Distribution Consistency based Graph Self-training
• Graph Size-imbalanced Learning with Energy-guided Structural Smoothing