Domain Adaptation for Statistical Classifiers

Abstract

The most basic assumption used in statistical learning theory is that training data and test data are drawn from the same underlying distribution. Unfortunately, in many applications, the "in-domain" test data is drawn from a distribution that is related, but not identical, to the "out-of-domain" distribution of the training data. We consider the common case in which labeled out-of-domain data is plentiful, but labeled in-domain data is scarce. We introduce a statistical formulation of this problem in terms of a simple mixture model and present an instantiation of this framework to maximum entropy classifiers and their linear chain counterparts. We present efficient inference algorithms for this special case based on the technique of conditional expectation maximization. Our experimental results show that our approach leads to improved performance on three real world tasks on four different data sets from the natural language processing domain.

Cited by in corpus (90)

• A Survey of Deep Learning Techniques for Autonomous Driving
• Frustratingly Easy Domain Adaptation
• A review of domain adaptation without target labels
• Beyond Sharing Weights for Deep Domain Adaptation
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Learning Representations for Counterfactual Inference
• Learn on Source, Refine on Target:A Model Transfer Learning Framework with Random Forests
• Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation
• Representation Learning: A Review and New Perspectives
• Knowledge Transfer between Buildings for Seismic Damage Diagnosis through Adversarial Learning
• Visuo-Haptic Object Perception for Robots: An Overview
• Multiple Source Adaptation and the Renyi Divergence
• Private Federated Learning with Domain Adaptation
• Improving Statistical Machine Translation for a Resource-Poor Language Using Related Resource-Rich Languages
• Efficient Optimization of Performance Measures by Classifier Adaptation
• Open Challenges in Synthetic Speech Detection
• Break-It-Fix-It: Unsupervised Learning for Program Repair
• Domain Adaptations for Computer Vision Applications
• Adversarial Balancing-based Representation Learning for Causal Effect Inference with Observational Data
• Which Clustering Do You Want? Inducing Your Ideal Clustering with Minimal Feedback
• Unknown Examples & Machine Learning Model Generalization
• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation
• Cross-Language Domain Adaptation for Classifying Crisis-Related Short Messages
• Reuse and Adaptation for Entity Resolution through Transfer Learning
• DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglement
• Refining Language Models with Compositional Explanations
• Libri-Adapt: A New Speech Dataset for Unsupervised Domain Adaptation
• Pre-Trained Models: Past, Present and Future
• Quantification in-the-wild: data-sets and baselines
• Predicting the Effectiveness of Self-Training: Application to Sentiment Classification
• Rethinking Importance Weighting for Deep Learning under Distribution Shift
• Cross-domain Semantic Parsing via Paraphrasing
• Transfer Learning for Thermal Comfort Prediction in Multiple Cities
• Robust Learning with the Hilbert-Schmidt Independence Criterion
• Friendship Prediction in Composite Social Networks
• Quantum subspace alignment for domain adaptation
• Kernelized Heterogeneous Risk Minimization
• Neural Structural Correspondence Learning for Domain Adaptation
• Improving Robustness of Deep Learning Based Knee MRI Segmentation: Mixup and Adversarial Domain Adaptation
• Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling
• Dangers of Bayesian Model Averaging under Covariate Shift
• On generalization in moment-based domain adaptation
• Strategies for Conceptual Change in Convolutional Neural Networks
• Learn to Expect the Unexpected: Probably Approximately Correct Domain Generalization
• An Extended Framework for Marginalized Domain Adaptation
• Estimating the Brittleness of AI: Safety Integrity Levels and the Need for Testing Out-Of-Distribution Performance
• CrossTrainer: Practical Domain Adaptation with Loss Reweighting
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Driving Experience Transfer Method for End-to-End Control of Self-Driving Cars
• Predicting the Future Behavior of a Time-Varying Probability Distribution
• Workshop on Quantification, Communication, and Interpretation of Uncertainty in Simulation and Data Science
• Domain Adaptation with Optimal Transport on the Manifold of SPD matrices
• A Survey on Spatial and Spatiotemporal Prediction Methods
• Compositional Dictionaries for Domain Adaptive Face Recognition
• Towards Accurate Word Segmentation for Chinese Patents
• PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models
• Joint cross-domain classification and subspace learning for unsupervised adaptation
• Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistency
• Learning to classify with possible sensor failures
• Pre-train or Annotate? Domain Adaptation with a Constrained Budget
• Domain Transfer in Dialogue Systems without Turn-Level Supervision
• ET-GAN: Cross-Language Emotion Transfer Based on Cycle-Consistent Generative Adversarial Networks
• Towards a continuous modeling of natural language domains
• Factorial Hidden Markov Models for Learning Representations of Natural Language
• Contrastive Language Adaptation for Cross-Lingual Stance Detection
• Domain Adaptation: Overfitting and Small Sample Statistics
• Predicting Ground-Level Scene Layout from Aerial Imagery
• A Zero-Shot Learning application in Deep Drawing process using Hyper-Process Model
• Balancing Method for High Dimensional Causal Inference
• Online Updating of Word Representations for Part-of-Speech Tagging
• Representation as a Service
• LM-Critic: Language Models for Unsupervised Grammatical Error Correction
• MEG Decoding Across Subjects
• Online Bagging for Anytime Transfer Learning
• Towards Robust Pattern Recognition: A Review
• $n$-Reference Transfer Learning for Saliency Prediction
• Discriminative Density-ratio Estimation
• Weakly Supervised Domain Detection
• Domain Adaptive Bootstrap Aggregating
• Causality and Generalizability: Identifiability and Learning Methods
• Causal Transfer Random Forest: Combining Logged Data and Randomized Experiments for Robust Prediction
• ReadOnce Transformers: Reusable Representations of Text for Transformers
• Conceptual Domain Adaptation Using Deep Learning
• Transfer Regression via Pairwise Similarity Regularization
• Distributionally Robust Learning with Stable Adversarial Training
• Mining User Profiles to Support Structure and Explanation in Open Social Networking
• Research Frontiers in Transfer Learning -- a systematic and bibliometric review
• Robust training on approximated minimal-entropy set
• Learning Synthetic to Real Transfer for Localization and Navigational Tasks
• Learning to Transfer: A Foliated Theory