1 citations · 1 across the 4 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
A Bayesian Model Selection Criterion for Selecting Pretraining Checkpoints
Michael Munn, Susan Wei
Recent advances in artificial intelligence have been fueled by the development of foundation models such as BERT, GPT, T5, and Vision Transformers. These models are first pretraine…
cs.LG2024★ 1 cited
Loss Landscape Degeneracy and Stagewise Development in Transformers
Jesse Hoogland, George Wang, Matthew Farrugia-Roberts +3
Deep learning involves navigating a high-dimensional loss landscape over the neural network parameter space. Over the course of training, complex computational structures form and…