Towards a universal neural network encoder for time series

Abstract

We study the use of a time series encoder to learn representations that are useful on data set types with which it has not been trained on. The encoder is formed of a convolutional neural network whose temporal output is summarized by a convolutional attention mechanism. This way, we obtain a compact, fixed-length representation from longer, variable-length time series. We evaluate the performance of the proposed approach on a well-known time series classification benchmark, considering full adaptation, partial adaptation, and no adaptation of the encoder to the new data type. Results show that such strategies are competitive with the state-of-the-art, often outperforming conceptually-matching approaches. Besides accuracy scores, the facility of adaptation and the efficiency of pre-trained encoders make them an appealing option for the processing of scarcely- or non-labeled time series.

arXiv comment10 pages, 2 figures

References in corpus (12)

• Scikit-learn: Machine Learning in Python
• Sequence to Sequence Learning with Neural Networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• mixup: Beyond Empirical Risk Minimization
• WaveNet: A Generative Model for Raw Audio
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Neural Discrete Representation Learning
• Universal Sentence Encoder
• Matching Networks for One Shot Learning
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder

Cited by in corpus (6)

• Transfer learning for time series classification
• Deep Neural Network Ensembles for Time Series Classification
• Time-Series Classification in Smart Manufacturing Systems: An Experimental Evaluation of State-of-the-Art Machine Learning Algorithms
• dCAM: Dimension-wise Class Activation Map for Explaining Multivariate Data Series Classification
• Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks
• Hardware-Friendly Delayed-Feedback Reservoir for Multivariate Time-Series Classification