Time-Series Classification in Smart Manufacturing Systems: An Experimental Evaluation of State-of-the-Art Machine Learning Algorithms

Abstract

Manufacturing is gathering extensive amounts of diverse data, thanks to the growing number of sensors and rapid advances in sensing technologies. Among the various data types available in SMS settings, time-series data plays a pivotal role. Hence, TSC emerges is crucial in this domain. The objective of this study is to fill this gap by providing a rigorous experimental evaluation of the SoTA ML and DL algorithms for TSC tasks in manufacturing and industrial settings. We first explored and compiled a comprehensive list of more than 92 SoTA algorithms from both TSC and manufacturing literature. Following, we selected the 36 most representative algorithms from this list. To evaluate their performance across various manufacturing classification tasks, we curated a set of 22 manufacturing datasets, representative of different characteristics that cover diverse manufacturing problems. Subsequently, we implemented and evaluated the algorithms on the manufacturing benchmark datasets, and analyzed the results for each dataset. Based on the results, ResNet, DrCIF, InceptionTime, and ARSENAL are the top-performing algorithms, boasting an average accuracy of over 96.6% across all 22 manufacturing TSC datasets. These findings underscore the robustness, efficiency, scalability, and effectiveness of convolutional kernels in capturing temporal features in time-series data, as three out of the top four performing algorithms leverage these kernels for feature extraction. Additionally, LSTM, BiLSTM, and TS-LSTM algorithms deserve recognition for their effectiveness in capturing features within time-series data using RNN-based structures.

Published in Robotics and Computer-Integrated Manufacturing journal

References in corpus (19)

• Generative Adversarial Networks
• Deep learning for time series classification: a review
• InceptionTime: Finding AlexNet for Time Series Classification
• Multivariate LSTM-FCNs for Time Series Classification
• ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels
• Benchmarking Multivariate Time Series Classification Algorithms
• Highly comparative feature-based time-series classification
• TS-CHIEF: A Scalable and Accurate Forest Algorithm for Time Series Classification
• Proximity Forest: An effective and scalable distance-based classifier for time series
• Should we really use post-hoc tests based on mean-ranks?
• Multivariate Time Series Classification with WEASEL+MUSE
• Towards a universal neural network encoder for time series
• Time-Series Pattern Recognition in Smart Manufacturing Systems: A Literature Review and Ontology
• Scalable Dictionary Classifiers for Time Series Classification
• A tale of two toolkits, report the third: on the usage and performance of HIVE-COTE v1.0
• Multivariate Time Series Classification using Dilated Convolutional Neural Network
• DTW-Merge: A Novel Data Augmentation Technique for Time Series Classification
• MrSQM: Fast Time Series Classification with Symbolic Representations
• Building Energy Efficiency through Advanced Regression Models and Metaheuristic Techniques for Sustainable Management

Cited by in corpus (3)

• Time-Series Forecasting in Smart Manufacturing Systems: An Experimental Evaluation of the State-of-the-art Algorithms
• Hybrid Agentic AI and Multi-Agent Systems in Smart Manufacturing
• Beyond Foundation Models: Dimension-Aware Neural Architecture Search with Small-Data Representation Models for Cryocooler Lifetime Prediction