Deep Efficient Continuous Manifold Learning for Time Series Modeling

Abstract

Modeling non-Euclidean data is drawing extensive attention along with the unprecedented successes of deep neural networks in diverse fields. Particularly, a symmetric positive definite matrix is being actively studied in computer vision, signal processing, and medical image analysis, due to its ability to learn beneficial statistical representations. However, owing to its rigid constraints, it remains challenging to optimization problems and inefficient computational costs, especially, when incorporating it with a deep learning framework. In this paper, we propose a framework to exploit a diffeomorphism mapping between Riemannian manifolds and a Cholesky space, by which it becomes feasible not only to efficiently solve optimization problems but also to greatly reduce computation costs. Further, for dynamic modeling of time-series data, we devise a continuous manifold learning method by systematically integrating a manifold ordinary differential equation and a gated recurrent neural network. It is worth noting that due to the nice parameterization of matrices in a Cholesky space, training our proposed network equipped with Riemannian geometric metrics is straightforward. We demonstrate through experiments over regular and irregular time-series datasets that our proposed model can be efficiently and reliably trained and outperforms existing manifold methods and state-of-the-art methods in various time-series tasks.

References in corpus (19)

• FaceNet: A Unified Embedding for Face Recognition and Clustering
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• WaveNet: A Generative Model for Raw Audio
• ShapeNet: An Information-Rich 3D Model Repository
• FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking
• DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG
• Multivariate LSTM-FCNs for Time Series Classification
• SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging
• Shrinkage Algorithms for MMSE Covariance Estimation
• Benchmarking Multivariate Time Series Classification Algorithms
• Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification
• XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging
• Intra- and Inter-epoch Temporal Context Network (IITNet) Using Sub-epoch Features for Automatic Sleep Scoring on Raw Single-channel EEG
• GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series
• Multivariate Time Series Classification with WEASEL+MUSE
• Riemannian Geometry of Symmetric Positive Definite Matrices via Cholesky Decomposition
• Gated recurrent units viewed through the lens of continuous time dynamical systems
• Second-order Convolutional Neural Networks
• Neural Ordinary Differential Equations on Manifolds