Stationary Density Estimation of Itô Diffusions Using Deep Learning

Abstract

In this paper, we consider the density estimation problem associated with the stationary measure of ergodic Itô diffusions from a discrete-time series that approximate the solutions of the stochastic differential equations. To take an advantage of the characterization of density function through the stationary solution of a parabolic-type Fokker-Planck PDE, we proceed as follows. First, we employ deep neural networks to approximate the drift and diffusion terms of the SDE by solving appropriate supervised learning tasks. Subsequently, we solve a steady-state Fokker-Plank equation associated with the estimated drift and diffusion coefficients with a neural-network-based least-squares method. We establish the convergence of the proposed scheme under appropriate mathematical assumptions, accounting for the generalization errors induced by regressing the drift and diffusion coefficients, and the PDE solvers. This theoretical study relies on a recent perturbation theory of Markov chain result that shows a linear dependence of the density estimation to the error in estimating the drift term, and generalization error results of nonparametric regression and of PDE regression solution obtained with neural-network models. The effectiveness of this method is reflected by numerical simulations of a two-dimensional Student's t distribution and a 20-dimensional Langevin dynamics.

References in corpus (32)

• Adam: A Method for Stochastic Optimization
• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Nonlinear Gluon Evolution in the Color Glass Condensate: I
• A Convergence Theory for Deep Learning via Over-Parameterization
• A Mean Field View of the Landscape of Two-Layers Neural Networks
• Weak Adversarial Networks for High-dimensional Partial Differential Equations
• Gradient Descent Provably Optimizes Over-parameterized Neural Networks
• Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains
• Deep Network Approximation for Smooth Functions
• Solving Fokker-Planck equation using deep learning
• Nonlinear mean field Fokker-Planck equations. Application to the chemotaxis of biological populations
• A Priori Estimates of the Population Risk for Two-layer Neural Networks
• Nonparametric Density Estimation for High-Dimensional Data - Algorithms and Applications
• SelectNet: Self-paced Learning for High-dimensional Partial Differential Equations
• Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth
• Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionality
• The phase diagram of approximation rates for deep neural networks
• Convergence Rate Analysis for Deep Ritz Method
• Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
• Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
• A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth
• A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations
• Deep Nonparametric Regression on Approximate Manifolds: Non-Asymptotic Error Bounds with Polynomial Prefactors
• Characterization of the Variation Spaces Corresponding to Shallow Neural Networks
• A deep learning method for solving Fokker-Planck equations
• Neural network approximation and estimation of classifiers with classification boundary in a Barron class
• On the Representation of Solutions to Elliptic PDEs in Barron Spaces
• Solving PDEs on Unknown Manifolds with Machine Learning
• Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Itô Diffusions