Towards a regularity theory for ReLU networks -- chain rule and global error estimates
arXiv:1905.04992 · doi:10.1109/SampTA45681.2019.9031005
Abstract
Although for neural networks with locally Lipschitz continuous activation functions the classical derivative exists almost everywhere, the standard chain rule is in general not applicable. We will consider a way of introducing a derivative for neural networks that admits a chain rule, which is both rigorous and easy to work with. In addition we will present a method of converting approximation results on bounded domains to global (pointwise) estimates. This can be used to extend known neural network approximation theory to include the study of regularity properties. Of particular interest is the application to neural networks with ReLU activation function, where it contributes to the understanding of the success of deep learning methods for high-dimensional partial differential equations.
arXiv commentAccepted for presentation at SampTA 2019
References in corpus (8)
- DGM: A deep learning algorithm for solving partial differential equations
- Solving high-dimensional partial differential equations using deep learning
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
- Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equations
- Optimal approximation of continuous functions by very deep ReLU networks
- DNN Expression Rate Analysis of High-dimensional PDEs: Application to Option Pricing
- The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems