9 citations · 35 across the 23 of their papers we have counts for
26 papers
Asymptotic convexity of wide and shallow neural networks
Vivek Borkar, Parthe Pandit
For a simple model of shallow and wide neural networks, we show that the epigraph of its input-output map as a function of the network parameters approximates epigraph of a. convex…
Feature maps for the Laplacian kernel and its generalizations
Sudhendu Ahir, Parthe Pandit
Recent applications of kernel methods in machine learning have seen a renewed interest in the Laplacian kernel, due to its stability to the bandwidth hyperparameter in comparison t…
Fast training of large kernel models with delayed projections
Amirhesam Abedsoltan, Siyuan Ma, Parthe Pandit +1
Classical kernel machines have historically faced significant challenges in scaling to large datasets and model sizes--a key ingredient that has driven the success of neural networ…
Mirror Descent on Reproducing Kernel Banach Spaces
Akash Kumar, Mikhail Belkin, Parthe Pandit
Recent advances in machine learning have led to increased interest in reproducing kernel Banach spaces (RKBS) as a more general framework that extends beyond reproducing kernel Hil…
Eigenvectors of the De Bruijn Graph Laplacian: A Natural Basis for the Cut and Cycle Space
Anthony Philippakis, Neil Mallinar, Parthe Pandit +1
We study the Laplacian of the undirected De Bruijn graph over an alphabet of order . While the eigenvalues of this Laplacian were found in 1998 by Delorme and Tillich [1], a…
Universality of Kernel Random Matrices and Kernel Regression in the Quadratic Regime
Parthe Pandit, Zhichao Wang, Yizhe Zhu
Kernel ridge regression (KRR) is a popular class of machine learning models that has become an important tool for understanding deep learning. Much of the focus thus far has been o…
Emergence in non-neural models: grokking modular arithmetic via average gradient outer product
Neil Mallinar, Daniel Beaglehole, Libin Zhu +3
Neural networks trained to solve modular arithmetic tasks exhibit grokking, a phenomenon where the test accuracy starts improving long after the model achieves 100% training accura…
On the Nystrom Approximation for Preconditioning in Kernel Machines
Amirhesam Abedsoltan, Parthe Pandit, Luis Rademacher +1
Kernel methods are a popular class of nonlinear predictive models in machine learning. Scalable algorithms for learning kernel models need to be iterative in nature, but convergenc…
Mechanism of feature learning in convolutional neural networks
Daniel Beaglehole, Adityanarayanan Radhakrishnan, Parthe Pandit +1
Understanding the mechanism of how convolutional neural networks learn features from image data is a fundamental problem in machine learning and computer vision. In this work, we i…
Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks
Evan Becker, Parthe Pandit, Sundeep Rangan +1
Generative Adversarial Networks (GANs) are a popular formulation to train generative models for complex high dimensional data. The standard method for training GANs involves a grad…
Toward Large Kernel Models
Amirhesam Abedsoltan, Mikhail Belkin, Parthe Pandit
Recent studies indicate that kernel machines can often perform similarly or better than deep neural networks (DNNs) on small datasets. The interest in kernel machines has been addi…
Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features
Adityanarayanan Radhakrishnan, Daniel Beaglehole, Parthe Pandit +1
In recent years neural networks have achieved impressive results on many technological and scientific tasks. Yet, the mechanism through which these models automatically select feat…
Instability and Local Minima in GAN Training with Kernel Discriminators
Evan Becker, Parthe Pandit, Sundeep Rangan +1
Generative Adversarial Networks (GANs) are a widely-used tool for generative modeling of complex data. Despite their empirical success, the training of GANs is not fully understood…
Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting
Neil Mallinar, James B. Simon, Amirhesam Abedsoltan +3
The practical success of overparameterized neural networks has motivated the recent scientific study of interpolating methods, which perfectly fit their training data. Certain inte…
A note on Linear Bottleneck networks and their Transition to Multilinearity
Libin Zhu, Parthe Pandit, Mikhail Belkin
Randomly initialized wide neural networks transition to linear functions of weights as the width grows, in a ball of radius around initialization. A necessary condition for…
On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensions
Daniel Beaglehole, Mikhail Belkin, Parthe Pandit
``Benign overfitting'', the ability of certain algorithms to interpolate noisy training data and yet perform well out-of-sample, has been a topic of considerable recent interest. W…
Kernel Methods and Multi-layer Perceptrons Learn Linear Models in High Dimensions
Mojtaba Sahraee-Ardakan, Melikasadat Emami, Parthe Pandit +2
Empirical observation of high dimensional phenomena, such as the double descent behaviour, has attracted a lot of interest in understanding classical techniques such as kernel meth…
Implicit Bias of Linear RNNs
Melikasadat Emami, Mojtaba Sahraee-Ardakan, Parthe Pandit +2
Contemporary wisdom based on empirical studies suggests that standard recurrent neural networks (RNNs) do not perform well on tasks requiring long-term memory. However, precise rea…
Low-Rank Nonlinear Decoding of -ECoG from the Primary Auditory Cortex
Melikasadat Emami, Mojtaba Sahraee-Ardakan, Parthe Pandit +6
This paper considers the problem of neural decoding from parallel neural measurements systems such as micro-electrocorticography (-ECoG). In systems with large numbers of array…
Generalization Error of Generalized Linear Models in High Dimensions
Melikasadat Emami, Mojtaba Sahraee-Ardakan, Parthe Pandit +2
At the heart of machine learning lies the question of generalizability of learned rules over previously unseen data. While over-parameterized models based on neural networks are no…
Inference in Multi-Layer Networks with Matrix-Valued Unknowns
Parthe Pandit, Mojtaba Sahraee-Ardakan, Sundeep Rangan +2
We consider the problem of inferring the input and hidden variables of a stochastic multi-layer neural network from an observation of the output. The hidden variables in each layer…
Inference with Deep Generative Priors in High Dimensions
Parthe Pandit, Mojtaba Sahraee-Ardakan, Sundeep Rangan +2
Deep generative priors offer powerful models for complex-structured data, such as images, audio, and text. Using these priors in inverse problems typically requires estimating the…
High-Dimensional Bernoulli Autoregressive Process with Long-Range Dependence
Parthe Pandit, Mojtaba Sahraee-Ardakan, Arash A. Amini +2
We consider the problem of estimating the parameters of a multivariate Bernoulli process with auto-regressive feedback in the high-dimensional setting where the number of samples a…
Asymptotics of MAP Inference in Deep Networks
Parthe Pandit, Mojtaba Sahraee, Sundeep Rangan +1
Deep generative priors are a powerful tool for reconstruction problems with complex data such as images and text. Inverse problems using such models require solving an inference pr…