TimeNet: Pre-trained deep recurrent neural network for time series classification

Abstract

Inspired by the tremendous success of deep Convolutional Neural Networks as generic feature extractors for images, we propose TimeNet: a deep recurrent neural network (RNN) trained on diverse time series in an unsupervised manner using sequence to sequence (seq2seq) models to extract features from time series. Rather than relying on data from the problem domain, TimeNet attempts to generalize time series representation across domains by ingesting time series from several domains simultaneously. Once trained, TimeNet can be used as a generic off-the-shelf feature extractor for time series. The representations or embeddings given by a pre-trained TimeNet are found to be useful for time series classification (TSC). For several publicly available datasets from UCR TSC Archive and an industrial telematics sensor data from vehicles, we observe that a classifier learned over the TimeNet embeddings yields significantly better performance compared to (i) a classifier learned over the embeddings given by a domain-specific RNN, as well as (ii) a nearest neighbor classifier based on Dynamic Time Warping.

25th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2017, Bruges, Belgium

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Recurrent Neural Network Regularization
• LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection
• Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder
• Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder

Cited by in corpus (16)

• Deep learning for time series classification: a review
• Unsupervised Scalable Representation Learning for Multivariate Time Series
• A comparison of LSTM and GRU networks for learning symbolic sequences
• A Deep Learning Framework for Generation and Analysis of Driving Scenario Trajectories
• Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding
• Scoring and Assessment in Medical VR Training Simulators with Dynamic Time Series Classification
• Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks
• Predictive Auto-scaling with OpenStack Monasca
• Autowarp: Learning a Warping Distance from Unlabeled Time Series Using Sequence Autoencoders
• Fast ES-RNN: A GPU Implementation of the ES-RNN Algorithm
• Pattern Discovery in Time Series with Byte Pair Encoding
• T-Rep: Representation Learning for Time Series using Time-Embeddings
• Predicting the outcome of team movements -- Player time series analysis using fuzzy and deep methods for representation learning
• From Time Series to Euclidean Spaces: On Spatial Transformations for Temporal Clustering
• A Self-Supervised Framework for Function Learning and Extrapolation
• Gated Res2Net for Multivariate Time Series Analysis