407 citations · 420 across the 15 of their papers we have counts for
4 papers · 1 filter
An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants
Michael Crawshaw, Chirag Modi, Mingrui Liu +1
To define a steepest descent method over a neural network, we need to choose a norm for each layer, a way to aggregate these norms across layers, and whether to use normalization.…
Constant Stepsize Local GD for Logistic Regression: Acceleration by Instability
Michael Crawshaw, Blake Woodworth, Mingrui Liu
Existing analysis of Local (Stochastic) Gradient Descent for heterogeneous objectives requires stepsizes where is the communication interval, which ensures monotoni…
Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
Michael Crawshaw, Mingrui Liu
Recent results in non-convex stochastic optimization demonstrate the convergence of popular adaptive algorithms (e.g., AdaGrad) under the -smoothness condition, but the…
Local Steps Speed Up Local GD for Heterogeneous Distributed Logistic Regression
Michael Crawshaw, Blake Woodworth, Mingrui Liu
We analyze two variants of Local Gradient Descent applied to distributed logistic regression with heterogeneous, separable data and show convergence at the rate for l…