451 citations · 1.7k across the 73 of their papers we have counts for
15 papers · 1 filter
Inclusive Artificial Intelligence
Dilip Arumugam, Shi Dong, Benjamin Van Roy
Prevailing methods for assessing and comparing generative AIs incentivize responses that serve a hypothetical representative individual. Evaluating models in these terms presumes h…
An Information-Theoretic Analysis of Compute-Optimal Neural Scaling Laws
Hong Jun Jeon, Benjamin Van Roy
We study the compute-optimal trade-off between model and training data set sizes for large neural networks. Our result suggests a linear relation similar to that supported by the e…
Posterior Sampling for Continuing Environments
Wanqiao Xu, Shi Dong, Benjamin Van Roy
We develop an extension of posterior sampling for reinforcement learning (PSRL) that is suited for a continuing agent-environment interface and integrates naturally into agent desi…
Fine-Tuning Language Models via Epistemic Neural Networks
Ian Osband, Seyed Mohammad Asghari, Benjamin Van Roy +3
Language models often pre-train on large unsupervised text corpora, then fine-tune on additional task-specific data. However, typical fine-tuning schemes do not prioritize the exam…
On Rate-Distortion Theory in Capacity-Limited Cognition & Reinforcement Learning
Dilip Arumugam, Mark K. Ho, Noah D. Goodman +1
Throughout the cognitive-science literature, there is widespread agreement that decision-making agents operating in the real world do so under limited information-processing capabi…
Is Stochastic Gradient Descent Near Optimal?
Yifan Zhu, Hong Jun Jeon, Benjamin Van Roy
The success of neural networks over the past decade has established them as effective models for many relevant data generating processes. Statistical theory on neural networks indi…