2 citations · 3 across the 7 of their papers we have counts for
5 papers · 1 filter
Lower Bounds for Anytime Acceleration of Gradient Descent
Chung-En Tsai, Ilyas Fatkhullin, Liang Zhang +1
Recent work suggests that the convergence rate of gradient descent (GD) in smooth convex optimization can be significantly improved by employing large stepsizes that may violate th…
Global Solutions to Non-Convex Functional Constrained Problems with Hidden Convexity
Ilyas Fatkhullin, Niao He, Guanghui Lan +1
Constrained non-convex optimization is fundamentally challenging, as global solutions are generally intractable and constraint qualifications may not hold. However, in many applica…
From Gradient Clipping to Normalization for Heavy Tailed SGD
Florian Hübler, Ilyas Fatkhullin, Niao He
Recent empirical evidence indicates that many machine learning applications involve heavy-tailed gradient noise, which challenges the standard assumptions of bounded variance in st…
Taming Nonconvex Stochastic Mirror Descent with General Bregman Divergence
Ilyas Fatkhullin, Niao He
This paper revisits the convergence of Stochastic Mirror Descent (SMD) in the contemporary nonconvex optimization setting. Existing results for batch-free nonconvex SMD restrict th…
Stochastic Optimization under Hidden Convexity
Ilyas Fatkhullin, Niao He, Yifan Hu
In this work, we consider constrained stochastic optimization problems under hidden convexity, i.e., those that admit a convex reformulation via non-linear (but invertible) map $c(…