226 citations · 905 across the 19 of their papers we have counts for
Showing 2021Show all
3 papers · 1 filter
cs.LG2021★ 6 cited
Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers
Colin Wei, Yining Chen, Tengyu Ma
A common lens to theoretically study neural net architectures is to analyze the functions they can approximate. However, constructions from approximation theory may be unrealistic…
cs.LG2021★ 33 cited
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning
Colin Wei, Sang Michael Xie, Tengyu Ma
Pretrained language models have achieved state-of-the-art performance when adapted to a downstream NLP task. However, theoretical analysis of these models is scarce and challenging…
cs.LG2021★ 41 cited
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
Jeff Z. HaoChen, Colin Wei, Adrien Gaidon +1
Recent works in self-supervised learning have advanced the state-of-the-art by relying on the contrastive learning paradigm, which learns representations by pushing positive pairs,…