2.3k citations · 4.7k across the 88 of their papers we have counts for
16 papers · 1 filter
Tradeoffs Between Contrastive and Supervised Learning: An Empirical Study
Ananya Karthik, Mike Wu, Noah Goodman +1
Contrastive learning has made considerable progress in computer vision, outperforming supervised pretraining on a range of downstream datasets. However, is contrastive learning the…
Causal Distillation for Language Models
Zhengxuan Wu, Atticus Geiger, Josh Rozner +5
Distillation efforts have led to language models that are more compact and efficient without serious drops in performance. The standard approach to distillation trains a student mo…
Inducing Causal Structure for Interpretable Neural Networks
Atticus Geiger, Zhengxuan Wu, Hanson Lu +5
In many areas, we have well-founded insights about causal structure that would be useful to bring into our trained models while still allowing them to learn in a data-driven fashio…
Open-domain clarification question generation without question examples
Julia White, Gabriel Poesia, Robert Hawkins +2
An overarching goal of natural language processing is to enable machines to communicate seamlessly with humans. However, natural language can be ambiguous or unclear. In cases of u…
Calibrate your listeners! Robust communication-based training for pragmatic speakers
Rose E. Wang, Julia White, Jesse Mu +1
To be good conversational partners, natural language processing (NLP) systems should be trained to produce contextually useful utterances. Prior work has investigated training NLP…
Temperature as Uncertainty in Contrastive Learning
Oliver Zhang, Mike Wu, Jasmine Bayrooti +1
Contrastive learning has demonstrated great capability to learn representations without annotations, even outperforming supervised baselines. However, it still lacks important prop…