Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding

Abstract

Time series are often complex and rich in information but sparsely labeled and therefore challenging to model. In this paper, we propose a self-supervised framework for learning generalizable representations for non-stationary time series. Our approach, called Temporal Neighborhood Coding (TNC), takes advantage of the local smoothness of a signal's generative process to define neighborhoods in time with stationary properties. Using a debiased contrastive objective, our framework learns time series representations by ensuring that in the encoding space, the distribution of signals from within a neighborhood is distinguishable from the distribution of non-neighboring signals. Our motivation stems from the medical field, where the ability to model the dynamic nature of time series data is especially valuable for identifying, tracking, and predicting the underlying patients' latent states in settings where labeling data is practically impossible. We compare our method to recently developed unsupervised representation learning approaches and demonstrate superior performance on clustering and classification tasks for multiple datasets.

arXiv commentCamera-ready at ICLR 2021

References in corpus (11)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Deep learning for time series classification: a review
• Recent Trends in Deep Learning Based Natural Language Processing
• Learning to Generate Reviews and Discovering Sentiment
• A Theoretical Analysis of Contrastive Unsupervised Representation Learning
• Medical Concept Representation Learning from Electronic Health Records and its Application on Heart Failure Prediction
• TimeNet: Pre-trained deep recurrent neural network for time series classification
• Deep Temporal Clustering : Fully Unsupervised Learning of Time-Domain Features
• Improving Clinical Predictions through Unsupervised Time Series Representation Learning
• Forecasting Individualized Disease Trajectories using Interpretable Deep Learning

Cited by in corpus (3)

• SleepPriorCL: Contrastive Representation Learning with Prior Knowledge-based Positive Mining and Adaptive Temperature for Sleep Staging
• Contrastive Neural Processes for Self-Supervised Learning
• A Self-Supervised Framework for Function Learning and Extrapolation