Marginalized Denoising Autoencoders for Domain Adaptation

Abstract

Stacked denoising autoencoders (SDAs) have been successfully used to learn new representations for domain adaptation. Recently, they have attained record accuracy on standard benchmark tasks of sentiment analysis across different text domains. SDAs learn robust data representations by reconstruction, recovering original features from data that are artificially corrupted with noise. In this paper, we propose marginalized SDA (mSDA) that addresses two crucial limitations of SDAs: high computational cost and lack of scalability to high-dimensional features. In contrast to SDAs, our approach of mSDA marginalizes noise and thus does not require stochastic gradient descent or other optimization algorithms to learn parameters ? in fact, they are computed in closed-form. Consequently, mSDA, which can be implemented in only 20 lines of MATLAB^{TM}, significantly speeds up SDAs by two orders of magnitude. Furthermore, the representations learnt by mSDA are as effective as the traditional SDAs, attaining almost identical accuracies in benchmark tasks.

ICML2012

References in corpus (1)

• Frustratingly Easy Domain Adaptation

Cited by in corpus (62)

• Learning Transferable Features with Deep Adaptation Networks
• Deep Visual Domain Adaptation: A Survey
• Deep Learning based Recommender System: A Survey and New Perspectives
• A review of domain adaptation without target labels
• A Survey on Metric Learning for Feature Vectors and Structured Data
• An introduction to domain adaptation and transfer learning
• Domain-Adversarial Neural Networks
• A Comprehensive Survey on Transfer Learning
• Learning Domain-Invariant Subspace using Domain Features and Independence Maximization
• ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
• Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
• Deep Residual Correction Network for Partial Domain Adaptation
• On the Feasibility of Transfer-learning Code Smells using Deep Learning
• Unsupervised Domain Adaptation with Adversarial Residual Transform Networks
• Robust Unsupervised Domain Adaptation for Neural Networks via Moment Alignment
• One Detector to Rule Them All: Towards a General Deepfake Attack Detection Framework
• Multi-source Domain Adaptation in the Deep Learning Era: A Systematic Survey
• Sample-to-Sample Correspondence for Unsupervised Domain Adaptation
• CNN-LSTM and Transfer Learning Models for Malware Classification based on Opcodes and API Calls
• Multi-modal Deep Analysis for Multimedia
• Hierarchically Learned View-Invariant Representations for Cross-View Action Recognition
• Early Action Prediction with Generative Adversarial Networks
• A New PAC-Bayesian Perspective on Domain Adaptation
• Domain Aggregation Networks for Multi-Source Domain Adaptation
• An alternative text representation to TF-IDF and Bag-of-Words
• Easy Transfer Learning By Exploiting Intra-domain Structures
• Semi-supervised representation learning via dual autoencoders for domain adaptation
• Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization
• Domain Adaptation Principal Component Analysis: base linear method for learning with out-of-distribution data
• ADAPT : Awesome Domain Adaptation Python Toolbox
• Multinomial Adversarial Networks for Multi-Domain Text Classification
• Cross-Domain Labeled LDA for Cross-Domain Text Classification
• PAC-Bayes and Domain Adaptation
• A Survey on Bayesian Deep Learning
• Deep Collective Matrix Factorization for Augmented Multi-View Learning
• Heterogeneous Domain Adaptation with Adversarial Neural Representation Learning: Experiments on E-Commerce and Cybersecurity
• Distributed Learning and Inference with Compressed Images
• Attention Regularized Laplace Graph for Domain Adaptation
• Building Surrogate Models of Nuclear Density Functional Theory with Gaussian Processesand Autoencoders
• Fine-grained Event Categorization with Heterogeneous Graph Convolutional Networks
• Bayesian Dropout
• Aggregating From Multiple Target-Shifted Sources
• Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation
• Multi-source Attention for Unsupervised Domain Adaptation
• Moment-Based Domain Adaptation: Learning Bounds and Algorithms
• Autoencoder based Domain Adaptation for Speaker Recognition under Insufficient Channel Information
• PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models
• Hierarchical Invariant Feature Learning with Marginalization for Person Re-Identification
• Transfer Learning Toolkit: Primers and Benchmarks
• CUDA: Contradistinguisher for Unsupervised Domain Adaptation
• Uncertainty-guided Model Generalization to Unseen Domains
• Domain-Invariant Feature Distillation for Cross-Domain Sentiment Classification
• Curriculum CycleGAN for Textual Sentiment Domain Adaptation with Multiple Sources
• Subdomain Adaptation with Manifolds Discrepancy Alignment
• Conditional Coupled Generative Adversarial Networks for Zero-Shot Domain Adaptation
• Autoencoding Improves Pre-trained Word Embeddings
• DILBERT: Customized Pre-Training for Domain Adaptation withCategory Shift, with an Application to Aspect Extraction
• Unsupervised Domain Adaptation with Global and Local Graph Neural Networks in Limited Labeled Data Scenario: Application to Disaster Management
• Streaming Social Event Detection and Evolution Discovery in Heterogeneous Information Networks
• CHEER: Rich Model Helps Poor Model via Knowledge Infusion
• Discriminative Embedding Autoencoder with a Regressor Feedback for Zero-Shot Learning
• Transferable End-to-End Aspect-based Sentiment Analysis with Selective Adversarial Learning