292 citations · 770 across the 46 of their papers we have counts for
33 papers · 1 filter
Generalization Bounds for Transformer-Based Next-Token Prediction in a Language Model
Insung Kong, Niklas Dexheimer, Johannes Schmidt-Hieber
A refined statistical understanding of LLM pre-training requires the analysis of the transformer architecture for data distributions that encapsulate key characteristics of text da…
Ordinal Patterns Based Change Points Detection
Annika Betken, Giorgio Micali, Johannes Schmidt-Hieber
The ordinal patterns of a fixed number of consecutive values in a time series is the spatial ordering of these values. Counting how often a specific ordinal pattern occurs in a tim…
Improving the Convergence Rates of Forward Gradient Descent with Repeated Sampling
Niklas Dexheimer, Johannes Schmidt-Hieber
Forward gradient descent (FGD) has been proposed as a biologically more plausible alternative of gradient descent as it can be computed without backward pass. Considering the linea…
Generative Modelling via Quantile Regression
Johannes Schmidt-Hieber, Petr Zamolodtchikov
We link conditional generative modelling to quantile regression. We propose a suitable loss function and derive minimax convergence rates for the associated risk under smoothness a…
Hebbian learning inspired estimation of the linear regression parameters from queries
Johannes Schmidt-Hieber, Wouter M Koolen
Local learning rules in biological neural networks (BNNs) are commonly referred to as Hebbian learning. [26] links a biologically motivated Hebbian learning rule to a specific zero…
Convergence guarantees for forward gradient descent in the linear regression model
Thijs Bos, Johannes Schmidt-Hieber
Renewed interest in the relationship between artificial and biological neural networks motivates the study of gradient-free methods. Considering the linear regression model with ra…
Dropout Regularization Versus -Penalization in the Linear Model
Gabriel Clara, Sophie Langer, Johannes Schmidt-Hieber
We investigate the statistical behavior of gradient descent iterates with dropout in the linear regression model. In particular, non-asymptotic bounds for the convergence of expect…
A supervised deep learning method for nonparametric density estimation
Thijs Bos, Johannes Schmidt-Hieber
Nonparametric density estimation is an unsupervised learning problem. In this work we propose a two-step procedure that casts the density estimation problem in the first step into…
Lower bounds for the trade-off between bias and mean absolute deviation
Alexis Derumigny, Johannes Schmidt-Hieber
In nonparametric statistics, rate-optimal estimators typically balance bias and stochastic error. The recent work on overparametrization raises the question whether rate-optimal es…
Codivergences and information matrices
Alexis Derumigny, Johannes Schmidt-Hieber
We propose a new concept of codivergence, which quantifies the similarity between two probability measures relative to a reference probability measure . In the neig…
A novel statistical approach to analyze image classification
Juntong Chen, Sophie Langer, Johannes Schmidt-Hieber
The recent statistical theory of neural networks focuses on nonparametric denoising problems that treat randomness as additive noise. Variability in image classification datasets d…
Local convergence rates of the nonparametric least squares estimator with applications to transfer learning
Johannes Schmidt-Hieber, Petr Zamolodtchikov
Convergence properties of empirical risk minimizers can be conveniently expressed in terms of the associated population risk. To derive bounds for the performance of the estimator…
Convergence rates of deep ReLU networks for multiclass classification
Thijs Bos, Johannes Schmidt-Hieber
For classification problems, trained deep neural networks return probabilities of class memberships. In this work we study convergence of the learned probabilities to the true cond…
Posterior contraction for deep Gaussian process priors
Gianluca Finocchio, Johannes Schmidt-Hieber
We study posterior contraction rates for a class of deep Gaussian process priors applied to the nonparametric regression problem under a general composition assumption on the regre…
On lower bounds for the bias-variance trade-off
Alexis Derumigny, Johannes Schmidt-Hieber
It is a common phenomenon that for high-dimensional and nonparametric statistical models, rate-optimal estimators balance squared bias and variance. Although this balancing is wide…
On frequentist coverage of Bayesian credible sets for estimation of the mean under constraints
Kevin Duisters, Johannes Schmidt-Hieber
Frequentist coverage of -highest posterior density (HPD) credible sets is studied in a signal plus noise model under a large class of noise distributions. We consider a spec…
Bayesian variance estimation in the Gaussian sequence model with partial information on the means
Gianluca Finocchio, Johannes Schmidt-Hieber
Consider the Gaussian sequence model under the additional assumption that a fixed fraction of the means is known. We study the problem of variance estimation from a frequentist Bay…
Nonparametric Bayesian analysis of the compound Poisson prior for support boundary recovery
Markus Reiss, Johannes Schmidt-Hieber
Given data from a Poisson point process with intensity frequentist properties for the Bayesian reconstruction of the support boundary func…
Posterior analysis of in the binomial problem with both parameters unknown -- with applications to quantitative nanoscopy
Johannes Schmidt-Hieber, Laura Fee Schneider, Thomas Staudt +3
Estimation of the population size from i.i.d.\ binomial observations with unknown success probability is relevant to a multitude of applications and has a long history.…
Asymptotic nonequivalence of density estimation and Gaussian white noise for small densities
Kolyan Ray, Johannes Schmidt-Hieber
It is well-known that density estimation on the unit interval is asymptotically equivalent to a Gaussian white noise experiment, provided the densities are sufficiently smooth and…
Nonparametric regression using deep neural networks with ReLU activation function
Johannes Schmidt-Hieber
Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly ch…
Posterior contraction rates for support boundary recovery
Markus Reiss, Johannes Schmidt-Hieber
Given a sample of a Poisson point process with intensity we study recovery of the boundary function from a nonparametric Bayes perspecti…
The Le Cam distance between density estimation, Poisson processes and Gaussian white noise
Kolyan Ray, Johannes Schmidt-Hieber
It is well-known that density estimation on the unit interval is asymptotically equivalent to a Gaussian white noise experiment, provided the densities have Hölder smoothness large…
Minimax theory for a class of non-linear statistical inverse problems
Kolyan Ray, Johannes Schmidt-Hieber
We study a class of statistical inverse problems with non-linear pointwise operators motivated by concrete statistical applications. A two-step procedure is proposed, where the fir…