160 citations · 173 across the 4 of their papers we have counts for
4 papers
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models
Hong Liu, Sang Michael Xie, Zhiyuan Li +1
Language modeling on large-scale datasets leads to impressive performance gains on various downstream language tasks. The validation pre-training loss (or perplexity in autoregress…
Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations
Jeff Z. HaoChen, Colin Wei, Ananya Kumar +1
Contrastive learning is a highly effective method for learning representations from unlabeled data. Recent works show that contrastive representations can transfer across domains,…
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution
Ananya Kumar, Aditi Raghunathan, Robbie Jones +2
When transferring a pretrained model to a downstream task, two popular methods are full fine-tuning (updating all the model parameters) and linear probing (updating only the last l…
Safe Reinforcement Learning by Imagining the Near Future
Garrett Thomas, Yuping Luo, Tengyu Ma
Safe reinforcement learning is a promising path toward applying reinforcement learning algorithms to real-world problems, where suboptimal behaviors may lead to actual negative con…