12 citations · 21 across the 20 of their papers we have counts for
3 papers · 1 filter
Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise
Enea Monzio Compagnoni, Tianlin Liu, Rustem Islamov +3
Despite the vast empirical evidence supporting the efficacy of adaptive optimization methods in deep learning, their theoretical understanding is far from complete. This work intro…
Loss Landscape Characterization of Neural Networks without Over-Parametrization
Rustem Islamov, Niccolò Ajroldi, Antonio Orvieto +1
Optimization methods play a crucial role in modern machine learning, powering the remarkable empirical achievements of deep learning models. These successes are even more remarkabl…
Towards Faster Decentralized Stochastic Optimization with Communication Compression
Rustem Islamov, Yuan Gao, Sebastian U. Stich
Communication efficiency has garnered significant attention as it is considered the main bottleneck for large-scale decentralized Machine Learning applications in distributed and f…