On-target Adaptation

Abstract

Domain adaptation seeks to mitigate the shift between training on the \emph{source} domain and testing on the \emph{target} domain. Most adaptation methods rely on the source data by joint optimization over source data and target data. Source-free methods replace the source data with a source model by fine-tuning it on target. Either way, the majority of the parameter updates for the model representation and the classifier are derived from the source, and not the target. However, target accuracy is the goal, and so we argue for optimizing as much as possible on the target data. We show significant improvement by on-target adaptation, which learns the representation purely from target data while taking only the source predictions for supervision. In the long-tailed classification setting, we show further improvement by on-target class distribution learning, which learns the (im)balance of classes from target data.

References in corpus (17)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Distilling the Knowledge in a Neural Network
• Bootstrap your own latent: A new approach to self-supervised Learning
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Temporal Ensembling for Semi-Supervised Learning
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• VisDA: The Visual Domain Adaptation Challenge
• Adversarial Discriminative Domain Adaptation
• Exploring Simple Siamese Representation Learning
• ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
• Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation
• Class-Balanced Loss Based on Effective Number of Samples
• Large-Scale Long-Tailed Recognition in an Open World
• Striking the Right Balance with Uncertainty
• Max-margin Class Imbalanced Learning with Gaussian Affinity