Unsupervised Scalable Representation Learning for Multivariate Time Series

Abstract

Time series constitute a challenging data type for machine learning algorithms, due to their highly variable lengths and sparse labeling in practice. In this paper, we tackle this challenge by proposing an unsupervised method to learn universal embeddings of time series. Unlike previous works, it is scalable with respect to their length and we demonstrate the quality, transferability and practicability of the learned representations with thorough experiments and comparisons. To this end, we combine an encoder based on causal dilated convolutions with a novel triplet loss employing time-based negative sampling, obtaining general-purpose representations for variable length and multivariate time series.

References in corpus (16)

• Distributed Representations of Words and Phrases and their Compositionality
• Representation Learning with Contrastive Predictive Coding
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• WaveNet: A Generative Model for Raw Audio
• Deep learning for time series classification: a review
• Unsupervised Learning of Video Representations using LSTMs
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• Recent Trends in Deep Learning Based Natural Language Processing
• Multi-Scale Convolutional Neural Networks for Time Series Classification
• An efficient framework for learning sentence representations
• Decomposing Motion and Content for Natural Video Sequence Prediction
• Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
• TimeNet: Pre-trained deep recurrent neural network for time series classification
• The UEA multivariate time series classification archive, 2018
• SOM-VAE: Interpretable Discrete Representation Learning on Time Series
• Random Warping Series: A Random Features Method for Time-Series Embedding

Cited by in corpus (15)

• Self-Supervised Representation Learning: Introduction, Advances and Challenges
• Deep Learning for Time Series Forecasting: Tutorial and Literature Survey
• Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Series
• COCOA: Cross Modality Contrastive Learning for Sensor Data
• Convolution-enhanced Evolving Attention Networks
• A Time-Temperature Dataset for the Strawberry Cold Chain Across Multiple Shipments and Locations
• A Scalable and Transferable Time Series Prediction Framework for Demand Forecasting
• Feature Selection for Multivariate Time Series via Network Pruning
• Contrastive Neural Processes for Self-Supervised Learning
• Unsupervised Visual Time-Series Representation Learning and Clustering
• Robust Group Anomaly Detection for Quasi-Periodic Network Time Series
• Unsupervised Driving Event Discovery Based on Vehicle CAN-data
• T-Rep: Representation Learning for Time Series using Time-Embeddings
• Leveraging Time Irreversibility with Order-Contrastive Pre-training
• Meta-Forecasting by combining Global Deep Representations with Local Adaptation