2 citations · 2 across the 7 of their papers we have counts for
4 papers · 1 filter
Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws
Zhiwei Xu, Shihao Wu, Hanseul Cho +2
Classical scaling laws for language model pretraining balance model size against training dataset size under a fixed compute budget, assuming abundant data and a single pass over t…
Last Layer Empirical Bayes
Valentin Villecroze, Yixin Wang, Gabriel Loaiza-Ganem
The task of quantifying the inherent uncertainty associated with neural network predictions is a key challenge in artificial intelligence. Bayesian neural networks (BNNs) and deep…
Deep Generative Models: Complexity, Dimensionality, and Approximation
Kevin Wang, Hongqian Niu, Yixin Wang +1
Generative networks have shown remarkable success in learning complex data distributions, particularly in generating high-dimensional data from lower-dimensional inputs. While this…
Deep Ensembles Secretly Perform Empirical Bayes
Gabriel Loaiza-Ganem, Valentin Villecroze, Yixin Wang
Quantifying uncertainty in neural networks is a highly relevant problem which is essential to many applications. The two predominant paradigms to tackle this task are Bayesian neur…