Showing math.STShow all
2 papers · 1 filter
math.ST2026
Generalization Bounds for Transformer-Based Next-Token Prediction in a Language Model
Insung Kong, Niklas Dexheimer, Johannes Schmidt-Hieber
A refined statistical understanding of LLM pre-training requires the analysis of the transformer architecture for data distributions that encapsulate key characteristics of text da…
math.ST2024
Improving the Convergence Rates of Forward Gradient Descent with Repeated Sampling
Niklas Dexheimer, Johannes Schmidt-Hieber
Forward gradient descent (FGD) has been proposed as a biologically more plausible alternative of gradient descent as it can be computed without backward pass. Considering the linea…