A Comprehensive Survey on Transfer Learning

Abstract

Transfer learning aims at improving the performance of target learners on target domains by transferring the knowledge contained in different but related source domains. In this way, the dependence on a large number of target domain data can be reduced for constructing target learners. Due to the wide application prospects, transfer learning has become a popular and promising area in machine learning. Although there are already some valuable and impressive surveys on transfer learning, these surveys introduce approaches in a relatively isolated way and lack the recent advances in transfer learning. Due to the rapid expansion of the transfer learning area, it is both necessary and challenging to comprehensively review the relevant studies. This survey attempts to connect and systematize the existing transfer learning researches, as well as to summarize and interpret the mechanisms and the strategies of transfer learning in a comprehensive way, which may help readers have a better understanding of the current research status and ideas. Unlike previous surveys, this survey paper reviews more than forty representative transfer learning approaches, especially homogeneous transfer learning approaches, from the perspectives of data and model. The applications of transfer learning are also briefly introduced. In order to show the performance of different transfer learning models, over twenty representative transfer learning models are used for experiments. The models are performed on three different datasets, i.e., Amazon Reviews, Reuters-21578, and Office-31. And the experimental results demonstrate the importance of selecting appropriate transfer learning models for different applications in practice.

31 pages, 7 figures

References in corpus (21)

• How transferable are features in deep neural networks?
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Deep Visual Domain Adaptation: A Survey
• Probabilistic Latent Semantic Analysis
• Conditional Adversarial Domain Adaptation
• Frustratingly Easy Domain Adaptation
• Unsupervised Domain Adaptation with Residual Transfer Networks
• A Survey on Multi-view Learning
• Marginalized Denoising Autoencoders for Domain Adaptation
• Asymmetric Tri-training for Unsupervised Domain Adaptation
• Aligning Domain-specific Distribution and Classifier for Cross-domain Classification from Multiple Sources
• Wasserstein Distance Guided Representation Learning for Domain Adaptation
• Multi-Representation Adaptation Network for Cross-domain Image Classification
• Learning with Augmented Features for Heterogeneous Domain Adaptation
• Learning Domain-Invariant Subspace using Domain Features and Independence Maximization
• Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
• Domain Agnostic Learning with Disentangled Representations
• Modeling Users' Behavior Sequences with Hierarchical Explainable Network for Cross-domain Fraud Detection
• Domain Adaptation with Category Attention Network for Deep Sentiment Analysis
• Transfer Learning Toolkit: Primers and Benchmarks

Cited by in corpus (42)

• Deep Subdomain Adaptation Network for Image Classification
• Multi-Task Learning with Deep Neural Networks: A Survey
• Deep learning for smart fish farming: applications, opportunities and challenges
• Review: Deep Learning in Electron Microscopy
• Transfer Learning as an Enabler of the Intelligent Digital Twin
• Forecasting and evaluating intervention of Covid-19 in the World
• Deep Learning for Identifying Iran's Cultural Heritage Buildings in Need of Conservation Using Image Classification and Grad-CAM
• Conservation Tools: The Next Generation of Engineering--Biology Collaborations
• Low-Cost Outdoor Air Quality Monitoring and Sensor Calibration: A Survey and Critical Analysis
• The PAU survey: Estimating galaxy photometry with deep learning
• Multi-task learning for virtual flow metering
• LabelAId: Just-in-time AI Interventions for Improving Human Labeling Quality and Domain Knowledge in Crowdsourcing Systems
• Self-Supervised Representation Learning for Detection of ACL Tear Injury in Knee MR Videos
• Trustworthy Convolutional Neural Networks: A Gradient Penalized-based Approach
• An Efficient Ground-aerial Transportation System for Pest Control Enabled by AI-based Autonomous Nano-UAVs
• Toward Efficient Transfer Learning in 6G
• Fast CRDNN: Towards on Site Training of Mobile Construction Machines
• Design and Implementation of A Novel Precision Irrigation Robot Based on An Intelligent Path Planning Algorithm
• A Study of Transfer Learning in Music Source Separation
• Transfer Learning Toolkit: Primers and Benchmarks
• Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records
• Hypothesis Disparity Regularized Mutual Information Maximization
• EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications
• Hyperparameter Transfer Across Developer Adjustments
• Multi-source Heterogeneous Domain Adaptation with Conditional Weighting Adversarial Network
• A Field Guide to Scientific XAI: Transparent and Interpretable Deep Learning for Bioinformatics Research
• Dynamic Knowledge Distillation for Black-box Hypothesis Transfer Learning
• Traffic Flow Estimation using LTE Radio Frequency Counters and Machine Learning
• An overall view of key problems in algorithmic trading and recent progress
• Open-Set Crowdsourcing using Multiple-Source Transfer Learning
• Reconstructing hadronically decaying tau leptons with a jet foundation model
• Transfer Learning as an Enhancement for Reconfiguration Management of Cyber-Physical Production Systems
• Can a powerful neural network be a teacher for a weaker neural network?
• Targeted Deep Learning: Framework, Methods, and Applications
• DWMD: Dimensional Weighted Orderwise Moment Discrepancy for Domain-specific Hidden Representation Matching
• Hybrid BYOL-ViT: Efficient approach to deal with small datasets
• Disentangling Transfer and Interference in Multi-Domain Learning
• NI-UDA: Graph Adversarial Domain Adaptation from Non-shared-and-Imbalanced Big Data to Small Imbalanced Applications
• Multiclass non-Adversarial Image Synthesis, with Application to Classification from Very Small Sample
• Domain Adaptor Networks for Hyperspectral Image Recognition
• Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation