Domain Adaptation for Statistical Classifiers

Abstract

The most basic assumption used in statistical learning theory is that training data and test data are drawn from the same underlying distribution. Unfortunately, in many applications, the "in-domain" test data is drawn from a distribution that is related, but not identical, to the "out-of-domain" distribution of the training data. We consider the common case in which labeled out-of-domain data is plentiful, but labeled in-domain data is scarce. We introduce a statistical formulation of this problem in terms of a simple mixture model and present an instantiation of this framework to maximum entropy classifiers and their linear chain counterparts. We present efficient inference algorithms for this special case based on the technique of conditional expectation maximization. Our experimental results show that our approach leads to improved performance on three real world tasks on four different data sets from the natural language processing domain.

Cited by in corpus (91)

• A Survey of Deep Learning Techniques for Autonomous Driving
• Frustratingly Easy Domain Adaptation
• A review of domain adaptation without target labels
• Domain Adaptation: Learning Bounds and Algorithms
• Beyond Sharing Weights for Deep Domain Adaptation
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Learning Representations for Counterfactual Inference
• Learn on Source, Refine on Target:A Model Transfer Learning Framework with Random Forests
• Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation
• Representation Learning: A Review and New Perspectives
• Knowledge Transfer between Buildings for Seismic Damage Diagnosis through Adversarial Learning
• Visuo-Haptic Object Perception for Robots: An Overview
• Multiple Source Adaptation and the Renyi Divergence
• Private Federated Learning with Domain Adaptation
• Improving Statistical Machine Translation for a Resource-Poor Language Using Related Resource-Rich Languages
• Efficient Optimization of Performance Measures by Classifier Adaptation
• Open Challenges in Synthetic Speech Detection
• Break-It-Fix-It: Unsupervised Learning for Program Repair
• Domain Adaptations for Computer Vision Applications
• Adversarial Balancing-based Representation Learning for Causal Effect Inference with Observational Data
• Which Clustering Do You Want? Inducing Your Ideal Clustering with Minimal Feedback
• Unknown Examples & Machine Learning Model Generalization
• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation
• Cross-Language Domain Adaptation for Classifying Crisis-Related Short Messages
• Reuse and Adaptation for Entity Resolution through Transfer Learning
• DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglement
• Refining Language Models with Compositional Explanations
• Libri-Adapt: A New Speech Dataset for Unsupervised Domain Adaptation
• Quantification in-the-wild: data-sets and baselines
• Pre-Trained Models: Past, Present and Future
• Predicting the Effectiveness of Self-Training: Application to Sentiment Classification
• Rethinking Importance Weighting for Deep Learning under Distribution Shift
• Cross-domain Semantic Parsing via Paraphrasing
• Transfer Learning for Thermal Comfort Prediction in Multiple Cities
• Quantum subspace alignment for domain adaptation
• Friendship Prediction in Composite Social Networks
• Robust Learning with the Hilbert-Schmidt Independence Criterion
• Kernelized Heterogeneous Risk Minimization
• Improving Robustness of Deep Learning Based Knee MRI Segmentation: Mixup and Adversarial Domain Adaptation
• Neural Structural Correspondence Learning for Domain Adaptation
• Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling
• On generalization in moment-based domain adaptation
• Dangers of Bayesian Model Averaging under Covariate Shift
• Strategies for Conceptual Change in Convolutional Neural Networks
• Learn to Expect the Unexpected: Probably Approximately Correct Domain Generalization
• Estimating the Brittleness of AI: Safety Integrity Levels and the Need for Testing Out-Of-Distribution Performance
• An Extended Framework for Marginalized Domain Adaptation
• CrossTrainer: Practical Domain Adaptation with Loss Reweighting
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Driving Experience Transfer Method for End-to-End Control of Self-Driving Cars
• Predicting the Future Behavior of a Time-Varying Probability Distribution
• Workshop on Quantification, Communication, and Interpretation of Uncertainty in Simulation and Data Science
• PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models
• A Survey on Spatial and Spatiotemporal Prediction Methods
• Compositional Dictionaries for Domain Adaptive Face Recognition
• Towards Accurate Word Segmentation for Chinese Patents
• Joint cross-domain classification and subspace learning for unsupervised adaptation
• Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistency
• Domain Adaptation with Optimal Transport on the Manifold of SPD matrices
• Pre-train or Annotate? Domain Adaptation with a Constrained Budget
• Domain Transfer in Dialogue Systems without Turn-Level Supervision
• ET-GAN: Cross-Language Emotion Transfer Based on Cycle-Consistent Generative Adversarial Networks
• Towards a continuous modeling of natural language domains
• Learning to classify with possible sensor failures
• Factorial Hidden Markov Models for Learning Representations of Natural Language
• Towards Robust Pattern Recognition: A Review
• Online Bagging for Anytime Transfer Learning
• Representation as a Service
• Domain Adaptation: Overfitting and Small Sample Statistics
• Contrastive Language Adaptation for Cross-Lingual Stance Detection
• MEG Decoding Across Subjects
• LM-Critic: Language Models for Unsupervised Grammatical Error Correction
• $n$-Reference Transfer Learning for Saliency Prediction
• Discriminative Density-ratio Estimation
• Online Updating of Word Representations for Part-of-Speech Tagging
• A Zero-Shot Learning application in Deep Drawing process using Hyper-Process Model
• Predicting Ground-Level Scene Layout from Aerial Imagery
• Balancing Method for High Dimensional Causal Inference
• Domain Adaptive Bootstrap Aggregating
• Transfer Regression via Pairwise Similarity Regularization
• Robust training on approximated minimal-entropy set
• Causal Transfer Random Forest: Combining Logged Data and Randomized Experiments for Robust Prediction
• Conceptual Domain Adaptation Using Deep Learning
• ReadOnce Transformers: Reusable Representations of Text for Transformers
• Distributionally Robust Learning with Stable Adversarial Training
• Mining User Profiles to Support Structure and Explanation in Open Social Networking
• Learning to Transfer: A Foliated Theory
• Causality and Generalizability: Identifiability and Learning Methods
• Weakly Supervised Domain Detection
• Learning Synthetic to Real Transfer for Localization and Navigational Tasks
• Research Frontiers in Transfer Learning -- a systematic and bibliometric review