3 papers
cs.LG2026
Robust Langevin Sampling under State-Dependent Heavy-Tailed Noise with Applications to LLM Fine-Tuning
M. Berk Sahin, Ege C. Kaya, Arda Fazla +2
Existing convergence guarantees for non-log-concave sampling with stochastic gradient Langevin Monte Carlo (SG-LMC) typically assume globally Lipschitz gradients and uniformly boun…
cs.LG2026
Why Does Adaptive Batching Help LLM Pretraining? A Perspective from Unbounded Variance
Arda Fazla, Antesh Upadhyay, Ege C. Kaya +2
Increasing the batch size during training is a common practice in large language model (LLM) pretraining, yet the theoretical justification behind its success is not well understoo…
cs.LG2026
A Banach-Space Theory of Markovian Halpern Iteration for Non-Expansive Maps
Ege C. Kaya, Arda Fazla, M. Berk Sahin +1
We study stochastic approximation of fixed points of a non-expansive operator when the oracle samples originate from a continuing Markovian trajectory. A direct block-minibatch…