Learning representations for multivariate time series with missing data using Temporal Kernelized Autoencoders

Abstract

Learning compressed representations of multivariate time series (MTS) facilitates data analysis in the presence of noise and redundant information, and for a large number of variates and time steps. However, classical dimensionality reduction approaches are designed for vectorial data and cannot deal explicitly with missing values. In this work, we propose a novel autoencoder architecture based on recurrent neural networks to generate compressed representations of MTS. The proposed model can process inputs characterized by variable lengths and it is specifically designed to handle missing data. Our autoencoder learns fixed-length vectorial representations, whose pairwise similarities are aligned to a kernel function that operates in input space and that handles missing values. This allows to learn good representations, even in the presence of a significant amount of missing data. To show the effectiveness of the proposed approach, we evaluate the quality of the learned representations in several classification tasks, including those involving medical data, and we compare to other methods for dimensionality reduction. Successively, we design two frameworks based on the proposed architecture: one for imputing missing data and another for one-class classification. Finally, we analyze under what circumstances an autoencoder with recurrent layers can learn better compressed representations of MTS than feed-forward architectures.

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Unsupervised Learning of Video Representations using LSTMs
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• A Recurrent Latent Variable Model for Sequential Data
• A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
• LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection
• Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
• Investigating echo state networks dynamics by means of recurrence analysis
• A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data
• Noisy multi-label semi-supervised dimensionality reduction
• Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder
• MIDA: Multiple Imputation using Denoising Autoencoders
• An Unsupervised Multivariate Time Series Kernel Approach for Identifying Patients with Surgical Site Infection from Blood Samples

Cited by in corpus (2)

• Patient Similarity Analysis with Longitudinal Health Data
• A Causal-based Framework for Multimodal Multivariate Time Series Validation Enhanced by Unsupervised Deep Learning as an Enabler for Industry 4.0