2.3k citations · 2.7k across the 29 of their papers we have counts for
4 papers · 1 filter
Task Ambiguity in Humans and Language Models
Alex Tamkin, Kunal Handa, Avash Shrestha +1
Language models have recently achieved strong performance across a wide range of NLP benchmarks. However, unlike benchmarks, real world tasks are often poorly specified, and agents…
Feature Dropout: Revisiting the Role of Augmentations in Contrastive Learning
Alex Tamkin, Margalit Glasgow, Xiluo He +1
What role do augmentations play in contrastive learning? Recent work suggests that good augmentations are label-preserving with respect to a specific downstream task. We complicate…
Active Learning Helps Pretrained Models Learn the Intended Task
Alex Tamkin, Dat Nguyen, Salil Deshpande +2
Models can fail in unpredictable ways during deployment due to task ambiguity, when multiple behaviors are consistent with the provided training data. An example is an object class…
Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies
Zhengxuan Wu, Alex Tamkin, Isabel Papadimitriou
When we transfer a pretrained language model to a new language, there are many axes of variation that change at once. To disentangle the impact of different factors like syntactic…