34 citations · 76 across the 12 of their papers we have counts for
13 papers
Flatness and Generalization: Learning Multi-Index Models with Homogeneous Neural Networks
Harsh Vardhan, Hossein Taheri, Arya Mazumdar
A common heuristic used to explain the generalization of first-order gradient methods on non-convex neural networks is that "flat interpolators generalize well" (Hochreiter and Sch…
On the Theory of Continual Learning with Gradient Descent for Neural Networks
Hossein Taheri, Avishek Ghosh, Arya Mazumdar
Continual learning, the ability of a model to adapt to an ongoing sequence of tasks without forgetting earlier ones, is a central goal of artificial intelligence. To better underst…
Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods
Hossein Taheri, Christos Thrampoulidis, Arya Mazumdar
In this paper, we study the data-dependent convergence and generalization behavior of gradient methods for neural networks with smooth activation. Our first result is a novel bound…
On the Optimization and Generalization of Multi-head Attention
Puneesh Deora, Rouzbeh Ghaderi, Hossein Taheri +1
The training and generalization dynamics of the Transformer's core mechanism, namely the Attention mechanism, remain under-explored. Besides, existing analyses primarily focus on s…
Fast Convergence in Learning Two-Layer Neural Networks with Separable Data
Hossein Taheri, Christos Thrampoulidis
Normalized gradient descent has shown substantial success in speeding up the convergence of exponentially-tailed loss functions (which includes exponential and logistic losses) on…
Generalization and Stability of Interpolating Neural Networks with Minimal Width
Hossein Taheri, Christos Thrampoulidis
We investigate the generalization and optimization properties of shallow neural-network classifiers trained by gradient descent in the interpolating regime. Specifically, in a real…
On Generalization of Decentralized Learning with Separable Data
Hossein Taheri, Christos Thrampoulidis
Decentralized learning offers privacy and communication efficiency when data are naturally distributed among agents communicating over an underlying graph. Motivated by overparamet…
Asymptotic Behavior of Adversarial Training in Binary Classification
Hossein Taheri, Ramtin Pedarsani, Christos Thrampoulidis
It has been consistently reported that many machine learning models are susceptible to adversarial attacks i.e., small additive adversarial perturbations applied to data points can…
Fundamental Limits of Ridge-Regularized Empirical Risk Minimization in High Dimensions
Hossein Taheri, Ramtin Pedarsani, Christos Thrampoulidis
Empirical Risk Minimization (ERM) algorithms are widely used in a variety of estimation and prediction tasks in signal-processing and machine learning applications. Despite their p…
Sharp Asymptotics and Optimal Performance for Inference in Binary Models
Hossein Taheri, Ramtin Pedarsani, Christos Thrampoulidis
We study convex empirical risk minimization for high-dimensional inference in binary models. Our first result sharply predicts the statistical performance of such estimators in the…
Quantized Decentralized Stochastic Learning over Directed Graphs
Hossein Taheri, Aryan Mokhtari, Hamed Hassani +1
We consider a decentralized stochastic learning problem where data points are distributed among computing nodes communicating over a directed graph. As the model size gets large, d…
Sharp Guarantees for Solving Random Equations with One-Bit Information
Hossein Taheri, Ramtin Pedarsani, Christos Thrampoulidis
We study the performance of a wide class of convex optimization-based estimators for recovering a signal from corrupted one-bit measurements in high-dimensions. Our general result…
Robust and Communication-Efficient Collaborative Learning
Amirhossein Reisizadeh, Hossein Taheri, Aryan Mokhtari +2
We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively. It is well-known that decentralized o…