10 papers · 1 filter
Generalization Bounds for Transformer-Based Next-Token Prediction in a Language Model
Insung Kong, Niklas Dexheimer, Johannes Schmidt-Hieber
A refined statistical understanding of LLM pre-training requires the analysis of the transformer architecture for data distributions that encapsulate key characteristics of text da…
A novel statistical approach to analyze image classification
Juntong Chen, Sophie Langer, Johannes Schmidt-Hieber
The recent statistical theory of neural networks focuses on nonparametric denoising problems that treat randomness as additive noise. Variability in image classification datasets d…
Ordinal Patterns Based Change Points Detection
Annika Betken, Giorgio Micali, Johannes Schmidt-Hieber
The ordinal patterns of a fixed number of consecutive values in a time series is the spatial ordering of these values. Counting how often a specific ordinal pattern occurs in a tim…
Improving the Convergence Rates of Forward Gradient Descent with Repeated Sampling
Niklas Dexheimer, Johannes Schmidt-Hieber
Forward gradient descent (FGD) has been proposed as a biologically more plausible alternative of gradient descent as it can be computed without backward pass. Considering the linea…
Generative Modelling via Quantile Regression
Johannes Schmidt-Hieber, Petr Zamolodtchikov
We link conditional generative modelling to quantile regression. We propose a suitable loss function and derive minimax convergence rates for the associated risk under smoothness a…
Convergence guarantees for forward gradient descent in the linear regression model
Thijs Bos, Johannes Schmidt-Hieber
Renewed interest in the relationship between artificial and biological neural networks motivates the study of gradient-free methods. Considering the linear regression model with ra…