10 citations · 24 across the 7 of their papers we have counts for
7 papers
Generalization at the Edge of Stability
Mario Tuci, Caner Korkmaz, Umut Şimşekli +1
Training modern neural networks often relies on large learning rates, operating at the edge of stability, where the optimization dynamics exhibit oscillatory and chaotic behavior.…
Stability, Complexity and Data-Dependent Worst-Case Generalization Bounds
Mario Tuci, Lennart Bastian, Benjamin Dupuis +3
Providing generalization guarantees for stochastic optimization algorithms remains a key challenge in learning theory. Recently, numerous works demonstrated the impact of the geome…
Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping
Umut Şimşekli, Mert Gürbüzbalaban, Sinan Yıldırım +1
The injection of heavy-tailed noise into the iterates of stochastic gradient descent (SGD) has garnered growing interest in recent years due to its theoretical and empirical benefi…
Generalization Bounds with Data-dependent Fractal Dimensions
Benjamin Dupuis, George Deligiannidis, Umut Şimşekli
Providing generalization guarantees for modern neural networks has been a crucial task in statistical learning. Recently, several studies have attempted to analyze the generalizati…
Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descent
Soon Hoe Lim, Yijun Wan, Umut Şimşekli
Recent studies have shown that gradient descent (GD) can achieve improved generalization when its dynamics exhibits a chaotic behavior. However, to obtain the desired effect, the s…
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms
Alexander Camuto, George Deligiannidis, Murat A. Erdogdu +3
Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade. While recent work has illustrated that the d…
Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance
Hongjian Wang, Mert Gürbüzbalaban, Lingjiong Zhu +2
Recent studies have provided both empirical and theoretical evidence illustrating that heavy tails can emerge in stochastic gradient descent (SGD) in various scenarios. Such heavy…