124 citations · 207 across the 8 of their papers we have counts for
8 papers
Adversarial Robustness is at Odds with Lazy Training
Yunjuan Wang, Enayat Ullah, Poorya Mianjy +1
Recent works show that adversarial examples exist for random neural networks [Daniely and Schacham, 2020] and that these examples can be found using a single step of gradient ascen…
On Convergence and Generalization of Dropout Training
Poorya Mianjy, Raman Arora
We study dropout in two-layer neural networks with rectified linear unit (ReLU) activations. Under mild overparametrization and assuming that the limiting kernel can separate the d…
Dropout: Explicit Forms and Capacity Control
Raman Arora, Peter Bartlett, Poorya Mianjy +1
We investigate the capacity control provided by dropout in various machine learning problems. First, we study dropout for matrix completion, where it induces a data-dependent regul…
On Dropout and Nuclear Norm Regularization
Poorya Mianjy, Raman Arora
We give a formal and complete characterization of the explicit regularizer induced by dropout in deep linear networks with squared loss. We show that (a) the explicit regularizer i…
Streaming Kernel PCA with Random Features
Enayat Ullah, Poorya Mianjy, Teodor V. Marinov +1
We study the statistical and computational aspects of kernel principal component analysis using random Fourier features and show that under mild assumptions, f…
On the Implicit Bias of Dropout
Poorya Mianjy, Raman Arora, Rene Vidal
Algorithmic approaches endow deep learning systems with implicit bias that helps them generalize even in over-parametrized settings. In this paper, we focus on understanding such a…
Stochastic Approximation for Canonical Correlation Analysis
Raman Arora, Teodor V. Marinov, Poorya Mianjy +1
We propose novel first-order stochastic approximation algorithms for canonical correlation analysis (CCA). Algorithms presented are instances of inexact matrix stochastic gradient…
Understanding Deep Neural Networks with Rectified Linear Units
Raman Arora, Amitabh Basu, Poorya Mianjy +1
In this paper we investigate the family of functions representable by deep neural networks (DNN) with rectified linear units (ReLU). We give an algorithm to train a ReLU DNN with o…