On-target Adaptation

Abstract

Domain adaptation seeks to mitigate the shift between training on the \emph{source} domain and testing on the \emph{target} domain. Most adaptation methods rely on the source data by joint optimization over source data and target data. Source-free methods replace the source data with a source model by fine-tuning it on target. Either way, the majority of the parameter updates for the model representation and the classifier are derived from the source, and not the target. However, target accuracy is the goal, and so we argue for optimizing as much as possible on the target data. We show significant improvement by on-target adaptation, which learns the representation purely from target data while taking only the source predictions for supervision. In the long-tailed classification setting, we show further improvement by on-target class distribution learning, which learns the (im)balance of classes from target data.

References in corpus (35)

• SMOTE: Synthetic Minority Over-sampling Technique
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Distilling the Knowledge in a Neural Network
• Bootstrap your own latent: A new approach to self-supervised Learning
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Improved Baselines with Momentum Contrastive Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Temporal Ensembling for Semi-Supervised Learning
• Conditional Adversarial Domain Adaptation
• When Does Label Smoothing Help?
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• VisDA: The Visual Domain Adaptation Challenge
• Model Adaptation: Unsupervised Domain Adaptation without Source Data
• Adversarial Discriminative Domain Adaptation
• Exploring Simple Siamese Representation Learning
• Tent: Fully Test-time Adaptation by Entropy Minimization
• Learning Robust Global Representations by Penalizing Local Predictive Power
• Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
• Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
• ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
• Improving robustness against common corruptions by covariate shift adaptation
• Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation
• Class-Balanced Loss Based on Effective Number of Samples
• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• Pseudo-Labeling Curriculum for Unsupervised Domain Adaptation
• Large-Scale Long-Tailed Recognition in an Open World
• LogME: Practical Assessment of Pre-trained Models for Transfer Learning
• Striking the Right Balance with Uncertainty
• Max-margin Class Imbalanced Learning with Gaussian Affinity
• Learning Visual Representations for Transfer Learning by Suppressing Texture