A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions

Abstract

Automating the monitoring of industrial processes has the potential to enhance efficiency and optimize quality by promptly detecting abnormal events and thus facilitating timely interventions. Deep learning, with its capacity to discern non-trivial patterns within large datasets, plays a pivotal role in this process. Standard deep learning methods are suitable to solve a specific task given a specific type of data. During training, deep learning demands large volumes of labeled data. However, due to the dynamic nature of the industrial processes and environment, it is impractical to acquire large-scale labeled data for standard deep learning training for every slightly different case anew. Deep transfer learning offers a solution to this problem. By leveraging knowledge from related tasks and accounting for variations in data distributions, the transfer learning framework solves new tasks with little or even no additional labeled data. The approach bypasses the need to retrain a model from scratch for every new setup and dramatically reduces the labeled data requirement. This survey first provides an in-depth review of deep transfer learning, examining the problem settings of transfer learning and classifying the prevailing deep transfer learning methods. Moreover, we delve into applications of deep transfer learning in the context of a broad spectrum of time series anomaly detection tasks prevalent in primary industrial domains, e.g., manufacturing process monitoring, predictive maintenance, energy management, and infrastructure facility monitoring. We discuss the challenges and limitations of deep transfer learning in industrial contexts and conclude the survey with practical directions and actionable suggestions to address the need to leverage diverse time series data for anomaly detection in an increasingly dynamic production environment.

arXiv comment27 pages, 8 figures, 2 tables, published in IEEE Acess

References in corpus (28)

• Deep Learning in Neural Networks: An Overview
• Explaining and Harnessing Adversarial Examples
• Knowledge Distillation: A Survey
• A systematic study of the class imbalance problem in convolutional neural networks
• An Overview of Multi-Task Learning in Deep Neural Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• On the Opportunities and Risks of Foundation Models
• Deep Learning for Anomaly Detection: A Survey
• Domain Generalization: A Survey
• Self-supervised Learning: Generative or Contrastive
• Toward Supervised Anomaly Detection
• DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection
• MST-GAT: A Multimodal Spatial-Temporal Graph Attention Network for Time Series Anomaly Detection
• An introduction to domain adaptation and transfer learning
• Domain-Adversarial Neural Networks
• A Novel GAN-based Fault Diagnosis Approach for Imbalanced Industrial Time Series
• TAnoGAN: Time Series Anomaly Detection with Generative Adversarial Networks
• Cross-Subject Transfer Learning in Human Activity Recognition Systems using Generative Adversarial Networks
• Deep Transfer Network: Unsupervised Domain Adaptation
• Deep Transfer Learning for Industrial Automation: A Review and Discussion of New Techniques for Data-Driven Machine Learning
• Time Series Anomaly Detection Using Convolutional Neural Networks and Transfer Learning
• Anomaly Generation using Generative Adversarial Networks in Host Based Intrusion Detection
• Annotated History of Modern AI and Deep Learning
• Is it enough to optimize CNN architectures on ImageNet?
• Towards Network Traffic Monitoring Using Deep Transfer Learning
• Contrastive Learning for Unsupervised Domain Adaptation of Time Series
• End-to-End Deep Neural Networks and Transfer Learning for Automatic Analysis of Nation-State Malware
• Anomaly Detection through Transfer Learning in Agriculture and Manufacturing IoT Systems

Cited by in corpus (2)

• Neural Networks for Generating Better Local Optima in Topology Optimization
• Fault Detection in New Wind Turbines with Limited Data by Generative Transfer Learning