34 citations · 76 across the 12 of their papers we have counts for
5 papers · 1 filter
Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods
Hossein Taheri, Christos Thrampoulidis, Arya Mazumdar
In this paper, we study the data-dependent convergence and generalization behavior of gradient methods for neural networks with smooth activation. Our first result is a novel bound…
On the Optimization and Generalization of Multi-head Attention
Puneesh Deora, Rouzbeh Ghaderi, Hossein Taheri +1
The training and generalization dynamics of the Transformer's core mechanism, namely the Attention mechanism, remain under-explored. Besides, existing analyses primarily focus on s…
Fast Convergence in Learning Two-Layer Neural Networks with Separable Data
Hossein Taheri, Christos Thrampoulidis
Normalized gradient descent has shown substantial success in speeding up the convergence of exponentially-tailed loss functions (which includes exponential and logistic losses) on…
On Generalization of Decentralized Learning with Separable Data
Hossein Taheri, Christos Thrampoulidis
Decentralized learning offers privacy and communication efficiency when data are naturally distributed among agents communicating over an underlying graph. Motivated by overparamet…
Robust and Communication-Efficient Collaborative Learning
Amirhossein Reisizadeh, Hossein Taheri, Aryan Mokhtari +2
We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively. It is well-known that decentralized o…