Deep Regression for Repeated Measurements

Abstract

Nonparametric mean function regression with repeated measurements serves as a cornerstone for many statistical branches, such as longitudinal/panel/functional data analysis. In this work, we investigate this problem using fully connected deep neural network (DNN) estimators with flexible shapes. A novel theoretical framework allowing arbitrary sampling frequency is established by adopting empirical process techniques to tackle clustered dependence. We then consider the DNN estimators for Hölder target function and illustrate a key phenomenon, the phase transition in the convergence rate, inherent to repeated measurements and its connection to the curse of dimensionality. Furthermore, we study several examples with low intrinsic dimensions, including the hierarchical composition model, low-dimensional support set and anisotropic Hölder smoothness. We also obtain new approximation results and matching lower bounds to demonstrate the adaptivity of the DNN estimators for circumventing the curse of dimensionality. Simulations and real data examples are provided to support our theoretical findings and practical implications.

88 pages, 6 figures

References in corpus (21)

• 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization
• Local Rademacher complexities
• Deep Neural Networks for Estimation and Inference
• Provable approximation properties for deep neural networks
• Optimal approximation of continuous functions by very deep ReLU networks
• Local polynomial regression on unknown manifolds
• Optimal estimation of the mean function based on discretely sampled functional data: Phase transition
• Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality
• Deep Network Approximation Characterized by Number of Neurons
• Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions
• Deep ReLU network approximation of functions on a manifold
• Regression on manifolds: Estimation of the exterior derivative
• Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression
• Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space
• CovNet: Covariance Networks for Functional Data on Multidimensional Domains
• Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks
• Estimation of the Mean Function of Functional Data via Deep Neural Networks
• Sample Complexity of Neural Policy Mirror Descent for Policy Optimization on Low-Dimensional Manifolds
• Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories
• On Deep Generative Models for Approximation and Estimation of Distributions on Manifolds
• Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks