Domain Adaptation: Learning Bounds and Algorithms

Abstract

This paper addresses the general problem of domain adaptation which arises in a variety of applications where the distribution of the labeled sample available somewhat differs from that of the test data. Building on previous work by Ben-David et al. (2007), we introduce a novel distance between distributions, discrepancy distance, that is tailored to adaptation problems with arbitrary loss functions. We give Rademacher complexity bounds for estimating the discrepancy distance from finite samples for different loss functions. Using this distance, we derive novel generalization bounds for domain adaptation for a wide family of loss functions. We also present a series of novel adaptation bounds for large classes of regularization-based algorithms, including support vector machines and kernel ridge regression based on the empirical discrepancy. This motivates our analysis of the problem of minimizing the empirical discrepancy for various loss functions for which we also give novel algorithms. We report the results of preliminary experiments that demonstrate the benefits of our discrepancy minimization algorithms for domain adaptation.

12 pages, 4 figures

References in corpus (1)

• Domain Adaptation for Statistical Classifiers

Cited by in corpus (140)

• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Ensemble Adversarial Training: Attacks and Defenses
• Unsupervised Domain Adaptation with Residual Transfer Networks
• A review of domain adaptation without target labels
• A Survey on Metric Learning for Feature Vectors and Structured Data
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Adaptive Personalized Federated Learning
• Three Approaches for Personalization with Applications to Federated Learning
• Learning Robust Global Representations by Penalizing Local Predictive Power
• An introduction to domain adaptation and transfer learning
• Domain-Adversarial Neural Networks
• Bridging Theory and Algorithm for Domain Adaptation
• Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis
• Learning Representations for Counterfactual Inference
• Third-Person Imitation Learning
• Domain Generalization by Marginal Transfer Learning
• Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation
• Estimating individual treatment effect: generalization bounds and algorithms
• Deep Transfer Learning with Joint Adaptation Networks
• How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks
• Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice
• Robust Unsupervised Domain Adaptation for Neural Networks via Moment Alignment
• Multi-Adversarial Domain Adaptation
• Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammars
• LEEP: A New Measure to Evaluate Transferability of Learned Representations
• Joint Distribution Optimal Transportation for Domain Adaptation
• Geometric Dataset Distances via Optimal Transport
• Lifelong Teacher-Student Network Learning
• Partial Transfer Learning with Selective Adversarial Networks
• Learning Transferable Parameters for Unsupervised Domain Adaptation
• Security and Privacy Issues in Deep Learning
• Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment
• Perfect Match: A Simple Method for Learning Representations For Counterfactual Inference With Neural Networks
• Fast Rates by Transferring from Auxiliary Hypotheses
• Virtual Mixup Training for Unsupervised Domain Adaptation
• Support and Invertibility in Domain-Invariant Representations
• Concentration in unbounded metric spaces and algorithmic stability
• Learning to Transfer Examples for Partial Domain Adaptation
• Learning Weighted Representations for Generalization Across Designs
• Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects
• A New PAC-Bayesian Perspective on Domain Adaptation
• Partial Adversarial Domain Adaptation
• Multiple Source Domain Adaptation with Adversarial Training of Neural Networks
• Inherent Tradeoffs in Learning Fair Representations
• Transfer of Machine Learning Fairness across Domains
• Transfer from Multiple MDPs
• Domain Aggregation Networks for Multi-Source Domain Adaptation
• Robust Local Preserving and Global Aligning Network for Adversarial Domain Adaptation
• Semi-Supervised Domain Adaptation with Non-Parametric Copulas
• Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data
• Semi-supervised representation learning via dual autoencoders for domain adaptation
• Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization
• A survey on domain adaptation theory: learning bounds and theoretical guarantees
• Understanding Self-Training for Gradual Domain Adaptation
• PAC-Bayes and Domain Adaptation
• Bridging the Theoretical Bound and Deep Algorithms for Open Set Domain Adaptation
• Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation
• Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier
• Specialized federated learning using a mixture of experts
• Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural Networks
• Domain adaptation under structural causal models
• Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback
• Generalization Bounds for Neural Belief Propagation Decoders
• Estimating Generalization under Distribution Shifts via Domain-Invariant Representations
• Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation
• Discrepancy-Based Active Learning for Domain Adaptation
• Calibrated Surrogate Losses for Adversarially Robust Classification
• Domain Alignment with Triplets
• Online Adaptation to Label Distribution Shift
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach
• Weighted Meta-Learning
• DART: Domain-Adversarial Residual-Transfer Networks for Unsupervised Cross-Domain Image Classification
• Neural Inverse Knitting: From Images to Manufacturing Instructions
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis
• Causality and Batch Reinforcement Learning: Complementary Approaches To Planning In Unknown Domains
• Open Set Domain Adaptation: Theoretical Bound and Algorithm
• Adversarial Domain Adaptation with Domain Mixup
• To Split or Not to Split: The Impact of Disparate Treatment in Classification
• Supervised Metric Learning with Generalization Guarantees
• Adversarial Bi-Regressor Network for Domain Adaptive Regression
• Unsupervised Domain Adaptation Based on Source-guided Discrepancy
• Butterfly: One-step Approach towards Wildly Unsupervised Domain Adaptation
• A Theory of Multiple-Source Adaptation with Limited Target Labeled Data
• Learning Decomposed Representation for Counterfactual Inference
• Continuous Transfer Learning with Label-informed Distribution Alignment
• Learn to Expect the Unexpected: Probably Approximately Correct Domain Generalization
• Computational Ceramicology
• Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
• On Regularization Parameter Estimation under Covariate Shift
• PAC-Bayesian Theorems for Domain Adaptation with Specialization to Linear Classifiers
• Bi-Classifier Determinacy Maximization for Unsupervised Domain Adaptation
• New Analysis and Algorithm for Learning with Drifting Distributions
• Discriminative Adversarial Domain Adaptation
• Empirically Estimable Classification Bounds Based on a New Divergence Measure
• A Generalized Neyman-Pearson Criterion for Optimal Domain Adaptation
• On Localized Discrepancy for Domain Adaptation
• A Critical View of the Structural Causal Model
• Joint Semantic Domain Alignment and Target Classifier Learning for Unsupervised Domain Adaptation
• OTCE: A Transferability Metric for Cross-Domain Cross-Task Representations
• Moment-Based Domain Adaptation: Learning Bounds and Algorithms
• Safeguarded Dynamic Label Regression for Generalized Noisy Supervision
• Counterfactual Cross-Validation: Stable Model Selection Procedure for Causal Inference Models
• Joint cross-domain classification and subspace learning for unsupervised adaptation
• Relation Adversarial Network for Low Resource Knowledge Graph Completion
• Generalization Bounds for Representative Domain Adaptation
• An Information-theoretic Approach to Distribution Shifts
• Multi-task Domain Adaptation for Sequence Tagging
• Transferable Time-Series Forecasting under Causal Conditional Shift
• Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm
• New Metrics to Evaluate the Performance and Fairness of Personalized Federated Learning
• When is Importance Weighting Correction Needed for Covariate Shift Adaptation?
• Unsupervised Domain Adaptation across FMCW Radar Configurations Using Margin Disparity Discrepancy
• Unsupervised Domain Adaptation for Constraining Star Formation Histories
• Ranking Neural Checkpoints
• Towards Accurate and Robust Domain Adaptation under Noisy Environments
• Alleviating Privacy Attacks via Causal Learning
• Target Consistency for Domain Adaptation: when Robustness meets Transferability
• Statistical Learning under Nonstationary Mixing Processes
• A Lazy Approach for Efficient Index Learning
• Teacher-Student Consistency For Multi-Source Domain Adaptation
• Classification under Data Contamination with Application to Remote Sensing Image Mis-registration
• Practical Transferability Estimation for Image Classification Tasks
• G2R Bound: A Generalization Bound for Supervised Learning from GAN-Synthetic Data
• Weakly Supervised Recovery of Semantic Attributes
• Distribution-Based Categorization of Classifier Transfer Learning
• KL Guided Domain Adaptation
• How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?
• Causal Forecasting:Generalization Bounds for Autoregressive Models
• Variational Auto-Encoder Architectures that Excel at Causal Inference
• An Improvement to the Domain Adaptation Bound in a PAC-Bayesian context
• Cross-domain error minimization for unsupervised domain adaptation
• Deep Domain Adaptation under Deep Label Scarcity
• Conceptual Domain Adaptation Using Deep Learning
• Knowledge Graph Simple Question Answering for Unseen Domains
• Approximating a Target Distribution using Weight Queries
• Self-Taught Support Vector Machine
• Towards Uncovering the Intrinsic Data Structures for Unsupervised Domain Adaptation using Structurally Regularized Deep Clustering