31 citations · 98 across the 11 of their papers we have counts for
19 papers
Understanding Contrastive Learning Requires Incorporating Inductive Biases
Nikunj Saunshi, Jordan Ash, Surbhi Goel +5
Contrastive learning is a popular form of self-supervised learning that encourages augmentations (views) of the same input to have more similar representations compared to augmenta…
Statistical Estimation from Dependent Data
Yuval Dagan, Constantinos Daskalakis, Nishanth Dikkala +2
We consider a general statistical estimation problem wherein binary labels across different observations are not independent conditioned on their feature vectors, but dependent, ca…
Investigating the Role of Negatives in Contrastive Representation Learning
Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy +1
Noise contrastive learning is a popular technique for unsupervised representation learning. In this approach, a representation is obtained via reduction to supervised learning, whe…
Acceleration via Fractal Learning Rate Schedules
Naman Agarwal, Surbhi Goel, Cyril Zhang
In practical applications of iterative first-order optimization, the learning rate schedule remains notoriously difficult to understand and expensive to tune. We demonstrate the pr…
Tight Hardness Results for Training Depth-2 ReLU Networks
Surbhi Goel, Adam Klivans, Pasin Manurangsi +1
We prove several hardness results for training depth-2 neural networks with the ReLU activation function; these networks are simply weighted sums (that may include negative coeffic…
From Boltzmann Machines to Neural Networks and Back Again
Surbhi Goel, Adam Klivans, Frederic Koehler
Graphical models are powerful tools for modeling high-dimensional data, but learning graphical models in the presence of latent variables is well-known to be difficult. In this wor…