2.3k citations · 2.7k across the 29 of their papers we have counts for
3 papers · 1 filter
Language Through a Prism: A Spectral Approach for Multiscale Language Representations
Alex Tamkin, Dan Jurafsky, Noah Goodman
Language exhibits structure at different scales, ranging from subwords to words, sentences, paragraphs, and documents. To what extent do deep models capture information at these sc…
Viewmaker Networks: Learning Views for Unsupervised Representation Learning
Alex Tamkin, Mike Wu, Noah Goodman
Many recent methods for unsupervised representation learning train models to be invariant to different "views," or distorted versions of an input. However, designing these views re…
Investigating Transferability in Pretrained Language Models
Alex Tamkin, Trisha Singh, Davide Giovanardi +1
How does language model pretraining help transfer learning? We consider a simple ablation technique for determining the impact of each pretrained layer on transfer task performance…