Variational latent discrete representation for time series modelling

Abstract

Discrete latent space models have recently achieved performance on par with their continuous counterparts in deep variational inference. While they still face various implementation challenges, these models offer the opportunity for a better interpretation of latent spaces, as well as a more direct representation of naturally discrete phenomena. Most recent approaches propose to train separately very high-dimensional prior models on the discrete latent data which is a challenging task on its own. In this paper, we introduce a latent data model where the discrete state is a Markov chain, which allows fast end-to-end training. The performance of our generative model is assessed on a building management dataset and on the publicly available Electricity Transformer Dataset.

arXiv comment2023 IEEE Workshop on Statistical Signal Processing (SSP 2023), IEEE, Jul 2023, Hano{ï}, Vietnam

References in corpus (10)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Categorical Reparameterization with Gumbel-Softmax
• Neural Discrete Representation Learning
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Semi-Supervised Learning with Deep Generative Models
• Zero-Shot Text-to-Image Generation
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• Rao-Blackwellized particle smoothers for conditionally linear Gaussian models
• Particle rejuvenation of Rao-Blackwellized Sequential Monte Carlo smoothers for Conditionally Linear and Gaussian models
• Last layer state space model for representation learning and uncertainty quantification