2.3k citations · 2.3k across the 2 of their papers we have counts for
2 papers
cs.LG2021★ 2.3k cited
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…
cs.LG2021★ 4 cited
The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion
Daniel Kunin, Javier Sagastuy-Brena, Lauren Gillespie +4
In this work we explore the limiting dynamics of deep neural networks trained with stochastic gradient descent (SGD). As observed previously, long after performance has converged,…