Bake off redux: a review and experimental evaluation of recent time series classification algorithms

Abstract

In 2017, a research paper compared 18 Time Series Classification (TSC) algorithms on 85 datasets from the University of California, Riverside (UCR) archive. This study, commonly referred to as a `bake off', identified that only nine algorithms performed significantly better than the Dynamic Time Warping (DTW) and Rotation Forest benchmarks that were used. The study categorised each algorithm by the type of feature they extract from time series data, forming a taxonomy of five main algorithm types. This categorisation of algorithms alongside the provision of code and accessible results for reproducibility has helped fuel an increase in popularity of the TSC field. Over six years have passed since this bake off, the UCR archive has expanded to 112 datasets and there have been a large number of new algorithms proposed. We revisit the bake off, seeing how each of the proposed categories have advanced since the original publication, and evaluate the performance of newer algorithms against the previous best-of-category using an expanded UCR archive. We extend the taxonomy to include three new categories to reflect recent developments. Alongside the originally proposed distance, interval, shapelet, dictionary and hybrid based algorithms, we compare newer convolution and feature based algorithms as well as deep learning approaches. We introduce 30 classification datasets either recently donated to the archive or reformatted to the TSC format, and use these to further evaluate the best performing algorithm from each category. Overall, we find that two recently proposed algorithms, Hydra+MultiROCKET and HIVE-COTEv2, perform significantly better than other approaches on both the current and new TSC problems.

References in corpus (5)

• Time Warp Edit Distance with Stiffness Adjustment for Time Series Matching
• A Review and Evaluation of Elastic Distance Functions for Time Series Clustering
• The FreshPRINCE: A Simple Transformation Based Pipeline Time Series Classifier
• Random Dilated Shapelet Transform: A New Approach for Time Series Shapelets
• Classifying Kepler light curves for 12,000 A and F stars using supervised feature-based machine learning

Cited by in corpus (11)

• Time-Series Forecasting in Smart Manufacturing Systems: An Experimental Evaluation of the State-of-the-art Algorithms
• Unsupervised Feature Based Algorithms for Time Series Extrinsic Regression
• Convolutional and Deep Learning based techniques for Time Series Ordinal Classification
• Exploring Kolmogorov-Arnold Networks for Interpretable Time Series Classification
• Boosting Photon-Number-Resolved Detection Rates of Transition-Edge Sensors by Machine Learning
• DeviceScope: An Interactive App to Detect and Localize Appliance Patterns in Electricity Consumption Time Series
• Rock the KASBA: Blazingly Fast and Accurate Time Series Clustering
• A 1-D CNN inference engine for constrained platforms
• Using matrix-product states for time-series machine learning
• Enhancing Time Series Classification with Diversity-Driven Neural Network Ensembles
• The Density of Cross-Persistence Diagrams and Its Applications