56 citations · 62 across the 12 of their papers we have counts for
Showing 2022Show all
3 papers · 1 filter
cs.LG2022
An Information-Theoretic Analysis of Compute-Optimal Neural Scaling Laws
Hong Jun Jeon, Benjamin Van Roy
We study the compute-optimal trade-off between model and training data set sizes for large neural networks. Our result suggests a linear relation similar to that supported by the e…
cs.LG2022
Is Stochastic Gradient Descent Near Optimal?
Yifan Zhu, Hong Jun Jeon, Benjamin Van Roy
The success of neural networks over the past decade has established them as effective models for many relevant data generating processes. Statistical theory on neural networks indi…
cs.LG2022★ 1 cited
An Information-Theoretic Framework for Supervised Learning
Hong Jun Jeon, Yifan Zhu, Benjamin Van Roy
Each year, deep learning demonstrates new and improved empirical results with deeper and wider neural networks. Meanwhile, with existing theoretical frameworks, it is difficult to…