2.3k citations · 2.3k across the 9 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.LG2021★ 2.3k cited
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…
cs.CL2021★ 8 cited
Frequency-based Distortions in Contextualized Word Embeddings
Kaitlyn Zhou, Kawin Ethayarajh, Dan Jurafsky
How does word frequency in pre-training data affect the behavior of similarity metrics in contextualized BERT embeddings? Are there systematic ways in which some word relationships…