10 papers
User-friendly approximation theory for statistical regression
Felix Benning
The error of regression using noisy function values naturally decomposes into an approximation error from the choice of model space and stochastic error from the noise on the obser…
Correlated initialization of deep residual networks
Felix Benning, Ivan Nourdin, Giovanni Peccati
We study the large-depth behavior of residual networks whose weights are correlated across layers at initialization. Our results confirm and extend a conjecture of Marion et al. [2…
Functional Scaling Limits of Interpolated Correlated Random Walks in Hölder Topology
Felix Benning, Ivan Nourdin
We prove functional scaling limits for interpolated random walks whose increments are functions of a stationary Gaussian sequence. In this setting, the classical Dobrushin-Major-Ta…
A Distributional View of High Dimensional Optimization
Felix Benning
This PhD thesis presents a distributional view of optimization in place of a worst-case perspective. We motivate this view with an investigation of the failure point of classical o…
Schoenberg characterization of continuous non-stationary isotropic positive definite kernels
Felix Benning, Max David Schölpple
We characterize the continuous isotropic positive definite kernels on , where isotropy refers to invariance under the orthogonal group but not necessarily stat…
In almost all shallow analytic neural network optimization landscapes, efficient minimizers have strongly convex neighborhoods
Felix Benning, Steffen Dereich
Whether or not a local minimum of a cost function has a strongly convex neighborhood greatly influences the asymptotic convergence rate of optimizers. In this article, we rigorousl…
Measure Theory of Conditionally Independent Random Function Evaluation
Felix Benning
In sequential design strategies, common in geostatistics and Bayesian optimization, the selection of a new observation point of a random function is informed…
Gradient Span Algorithms Make Predictable Progress in High Dimension
Felix Benning, Leif Döring
We prove that all 'gradient span algorithms' have asymptotically deterministic behavior on scaled Gaussian random functions as the dimension tends to infinity. This is a functional…
Random Function Descent
Felix Benning, Leif Döring
Classical worst-case optimization theory neither explains the success of optimization in machine learning, nor does it help with step size selection. In this paper we demonstrate t…
High Dimensional Optimization through the Lens of Machine Learning
Felix Benning
This thesis reviews numerical optimization methods with machine learning problems in mind. Since machine learning models are highly parametrized, we focus on methods suited for hig…