Spectral Diffusion Processes
arXiv:2209.14125
The paper introduces diffusion models for stochastic processes by representing data in a spectral domain using kernels, truncating the spectral coefficients, and modeling them with standard finite-dimensional diffusion models, enabling valid process generation and conditional sampling.
Abstract
Diffusion models have proven to be a flexible and effective framework for modelling probability distributions on finite-dimensional spaces. However, many physical modelling problems such as time series are naturally described over function spaces. In this work we apply diffusion models to such stochastic processes. To do so we consider a spectral representation of the data, obtained using a kernel, thereby dissociating the stochastic part of the processes from their space-time structure. As a result, the stochasticity of the processes is entirely encoded in the spectral coefficients, which we truncate and model using standard finite-dimensional diffusion models. By truncating the representation in the spectral domain we ensure our resulting model defines valid stochastic processes, thereby naturally satisfying consistency and exchangeability criteria. Projecting our spectral diffusion models back to the original input space, we show that for any given marginals our approach corresponds to a diffusion model with correlated noise, with explicit covariance matrix given by the kernel. We demonstrate our method's effectiveness for modelling various multimodal datasets as well as conditional sampling by amortising our models with respect to a context set.
This version (v3) extends the previous workshop version (v2) with conditional sampling and theoretical results. Work carried out in 2022/23. V2 appeared in Score-based Methods Workshop at the 36th Conference on Neural Information Processing Systems (NeurIPS 2022)