2.3k citations · 4.6k across the 26 of their papers we have counts for
3 papers · 1 filter
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…
When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset
Lucia Zheng, Neel Guha, Brandon R. Anderson +2
While self-supervised learning has made rapid advances in natural language processing, it remains unclear when researchers should engage in resource-intensive domain-specific pretr…
An Information-Theoretic Perspective on Credit Assignment in Reinforcement Learning
Dilip Arumugam, Peter Henderson, Pierre-Luc Bacon
How do we formalize the challenge of credit assignment in reinforcement learning? Common intuition would draw attention to reward sparsity as a key contributor to difficult credit…