Optimal Nonparametric Inference via Deep Neural Network

Abstract

Deep neural network is a state-of-art method in modern science and technology. Much statistical literature have been devoted to understanding its performance in nonparametric estimation, whereas the results are suboptimal due to a redundant logarithmic sacrifice. In this paper, we show that such log-factors are not necessary. We derive upper bounds for the $L^2$ minimax risk in nonparametric estimation. Sufficient conditions on network architectures are provided such that the upper bounds become optimal (without log-sacrifice). Our proof relies on an explicitly constructed network estimator based on tensor product B-splines. We also derive asymptotic distributions for the constructed network and a relating hypothesis testing procedure. The testing procedure is further proven as minimax optimal under suitable network architectures.

References in corpus (5)

• On the Number of Linear Regions of Deep Neural Networks
• Optimal approximation of continuous functions by very deep ReLU networks
• Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality
• Benefits of depth in neural networks
• On Deep Instrumental Variables Estimate

Cited by in corpus (3)

• Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Network
• Calibrating multi-dimensional complex ODE from noisy data via deep neural networks
• Estimation of the Mean Function of Functional Data via Deep Neural Networks