2 citations · 4 across the 12 of their papers we have counts for
4 papers · 1 filter
Asymptotic Risk of Learning-Rate Schedules for Least-Squares Regression with Discretely Distributed Features
Noé Colomban, Francis Bach, Fabian Schaipp
We study the last iterate of stochastic gradient descent (SGD) on streaming least squares with discretely distributed features (Slock's model), which decouples the dynamics across…
How to Allocate Your Tokens? Scaling Laws with Training Steps and Batch Size
Fabian Schaipp
We propose a scaling law that takes into account model size and training data while explicitly splitting the latter into training steps and batch size (called three-term law). Fitt…
Sparse regression, classification, and microbial network estimation in QIIME2 with q2-classo and q2-gglasso
Oleg Vlasovets, Fabian Schaipp, Leo Simpson +3
Motivation: Statistical analysis of microbial count data derived from 16S rRNA or metagenomics sequencing poses unique challenges due to the sparse, compositional, and high-dimensi…
Step-Size Stability in Stochastic Optimization: A Theoretical Perspective
Fabian Schaipp, Robert M. Gower, Adrien Taylor
We present a theoretical analysis of stochastic optimization methods in terms of their sensitivity with respect to the step size. We identify a key quantity that, for each method,…