64 citations · 158 across the 12 of their papers we have counts for
12 papers
Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time
Arvind Mahankali, Jeff Z. Haochen, Kefan Dong +2
Despite recent theoretical progress on the non-convex optimization of two-layer neural networks, it is still an open question whether gradient descent on neural networks without un…
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou +4
Language models have been shown to exhibit positive scaling, where performance improves as models are scaled up in terms of size, compute, or data. In this work, we introduce NeQA,…
Diagnosing and Rectifying Vision Models using Language
Yuhui Zhang, Jeff Z. HaoChen, Shih-Cheng Huang +3
Recent multi-modal contrastive learning models have demonstrated the ability to learn an embedding space suitable for building strong vision classifiers, by leveraging the rich inf…
A Theoretical Study of Inductive Biases in Contrastive Learning
Jeff Z. HaoChen, Tengyu Ma
Understanding self-supervised learning is important but challenging. Previous theoretical works study the role of pretraining losses, and view neural networks as general black boxe…
Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations
Jeff Z. HaoChen, Colin Wei, Ananya Kumar +1
Contrastive learning is a highly effective method for learning representations from unlabeled data. Recent works show that contrastive representations can transfer across domains,…
Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation
Kendrick Shen, Robbie Jones, Ananya Kumar +4
We consider unsupervised domain adaptation (UDA), where labeled data from a source domain (e.g., photographs) and unlabeled data from a target domain (e.g., sketches) are used to l…
Amortized Proximal Optimization
Juhan Bae, Paul Vicol, Jeff Z. HaoChen +1
We propose a framework for online meta-optimization of parameters that govern optimization, called Amortized Proximal Optimization (APO). We first interpret various existing neural…
Self-supervised Learning is More Robust to Dataset Imbalance
Hong Liu, Jeff Z. HaoChen, Adrien Gaidon +1
Self-supervised learning (SSL) is a scalable way to learn general visual representations since it learns without labels. However, large-scale unlabeled datasets in the wild often h…
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
Jeff Z. HaoChen, Colin Wei, Adrien Gaidon +1
Recent works in self-supervised learning have advanced the state-of-the-art by relying on the contrastive learning paradigm, which learns representations by pushing positive pairs,…
Meta-learning Transferable Representations with a Single Target Domain
Hong Liu, Jeff Z. HaoChen, Colin Wei +1
Recent works found that fine-tuning and joint training---two popular approaches for transfer learning---do not always improve accuracy on downstream tasks. First, we aim to underst…
Shape Matters: Understanding the Implicit Bias of the Noise Covariance
Jeff Z. HaoChen, Colin Wei, Jason D. Lee +1
The noise in stochastic gradient descent (SGD) provides a crucial implicit regularization effect for training overparameterized models. Prior theoretical work largely focuses on sp…
Random Shuffling Beats SGD after Finite Epochs
Jeff Z. HaoChen, Suvrit Sra
A long-standing problem in the theory of stochastic gradient descent (SGD) is to prove that its without-replacement version RandomShuffle converges faster than the usual with-repla…