Deep composition of tensor-trains using squared inverse Rosenblatt transports

Abstract

Characterising intractable high-dimensional random variables is one of the fundamental challenges in stochastic computation. The recent surge of transport maps offers a mathematical foundation and new insights for tackling this challenge by coupling intractable random variables with tractable reference random variables. This paper generalises the functional tensor-train approximation of the inverse Rosenblatt transport recently developed by Dolgov et al. (Stat Comput 30:603--625, 2020) to a wide class of high-dimensional non-negative functions, such as unnormalised probability density functions. First, we extend the inverse Rosenblatt transform to enable the transport to general reference measures other than the uniform measure. We develop an efficient procedure to compute this transport from a squared tensor-train decomposition which preserves the monotonicity. More crucially, we integrate the proposed order-preserving functional tensor-train transport into a nested variable transformation framework inspired by the layered structure of deep neural networks. The resulting deep inverse Rosenblatt transport significantly expands the capability of tensor approximations and transport maps to random variables with complicated nonlinear interactions and concentrated density functions. We demonstrate the efficiency of the proposed approach on a range of applications in statistical learning and uncertainty quantification, including parameter estimation for dynamical systems and inverse problems constrained by partial differential equations.

arXiv commentFound Comput Math (2021)

References in corpus (17)

• Exchange Monte Carlo Method and Application to Spin Glass Simulations
• Variational Inference with Normalizing Flows
• Normalizing Flows for Probabilistic Modeling and Inference
• Bayesian Inference with Optimal Maps
• Alternating minimal energy methods for linear systems in higher dimensions. Part I: SPD systems
• Alternating minimal energy methods for linear systems in higher dimensions. Part II: Faster algorithm and application to nonsymmetric systems
• Residual Flows for Invertible Generative Modeling
• Likelihood-informed dimension reduction for nonlinear inverse problems
• Data-Driven Model Reduction for the Bayesian Solution of Inverse Problems
• An introduction to sampling via measure transport
• Spectral tensor-train decomposition
• Transport map accelerated Markov chain Monte Carlo
• Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction
• Rectangular maximum-volume submatrices and their applications
• A Stein variational Newton method
• Relaxing Bijectivity Constraints with Continuously Indexed Normalising Flows
• Variational Inference with Continuously-Indexed Normalizing Flows

Cited by in corpus (3)

• On the representation and learning of monotone triangular transport maps
• Variational Bayesian Optimal Experimental Design with Normalizing Flows
• Tensor product algorithms for inference of contact network from epidemiological data