2 citations · 5 across the 7 of their papers we have counts for
7 papers
The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning
Nikhil Ghosh, Spencer Frei, Wooseok Ha +1
In this work, we investigate the dynamics of stochastic gradient descent (SGD) when training a single-neuron autoencoder with linear or ReLU activation on orthogonal data. We show…
The Power of External Memory in Increasing Predictive Model Capacity
Cenk Baykal, Dylan J Cutler, Nishanth Dikkala +3
One way of introducing sparsity into deep networks is by attaching an external table of parameters that is sparsely looked up at different layers of the network. By storing the bul…
Alternating Updates for Efficient Transformers
Cenk Baykal, Dylan Cutler, Nishanth Dikkala +3
It has been well established that increasing scale in deep transformer networks leads to improved quality and performance. However, this increase in scale often comes with prohibit…
A Universal Trade-off Between the Model Size, Test Loss, and Training Loss of Linear Predictors
Nikhil Ghosh, Mikhail Belkin
In this work we establish an algorithm and distribution independent non-asymptotic trade-off between the model size, excess test loss, and training loss of linear predictors. Speci…
Deconstructing Distributions: A Pointwise Framework of Learning
Gal Kaplun, Nikhil Ghosh, Saurabh Garg +2
In machine learning, we traditionally evaluate the performance of a single model, averaged over a collection of test inputs. In this work, we propose a new approach: we measure the…
The Three Stages of Learning Dynamics in High-Dimensional Kernel Methods
Nikhil Ghosh, Song Mei, Bin Yu
To understand how deep learning works, it is crucial to understand the training dynamics of neural networks. Several interesting hypotheses about these dynamics have been made base…
Landmark Ordinal Embedding
Nikhil Ghosh, Yuxin Chen, Yisong Yue
In this paper, we aim to learn a low-dimensional Euclidean representation from a set of constraints of the form "item j is closer to item i than item k". Existing approaches for th…