Showing 2023Show all
3 papers · 1 filter
cs.LG2023
Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition
Zhongtian Chen, Edmund Lau, Jake Mendel +2
We investigate phase transitions in a Toy Model of Superposition (TMS) using Singular Learning Theory (SLT). We derive a closed formula for the theoretical loss and, in the case of…
stat.ML2023
The Local Learning Coefficient: A Singularity-Aware Complexity Measure
Edmund Lau, Zach Furman, George Wang +2
The Local Learning Coefficient (LLC) is introduced as a novel complexity measure for deep neural networks (DNNs). Recognizing the limitations of traditional complexity measures, th…
stat.ML2023
Variational Bayesian Neural Networks via Resolution of Singularities
Susan Wei, Edmund Lau
In this work, we advocate for the importance of singular learning theory (SLT) as it pertains to the theory and practice of variational inference in Bayesian neural networks (BNNs)…