7 citations · 24 across the 15 of their papers we have counts for
7 papers · 1 filter
A Spectral Theory of Grokking: Weight Decay induces Feature Learning
Lenz Pracher, Pascal de Jong, Oskar Lieshaus +2
In grokking an early fit to the training data separates from a much later improvement in generalization. During this delay, training can move from a fixed neural tangent kernel (NT…
Not All Explanations for Deep Learning Phenomena Are Equally Valuable
Alan Jeffares, Mihaela van der Schaar
Developing a better understanding of surprising or counterintuitive phenomena has constituted a significant portion of deep learning research in recent years. These include double…
An Introduction to Discrete Variational Autoencoders
Alan Jeffares, Liyuan Liu
Variational Autoencoders (VAEs) are well-established as a principled approach to probabilistic unsupervised learning with neural networks. Typically, an encoder network defines the…
Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond
Alan Jeffares, Alicia Curth, Mihaela van der Schaar
Deep learning sometimes appears to work in unexpected ways. In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate mo…
TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specialization
Alan Jeffares, Tennison Liu, Jonathan Crabbé +2
Despite their success with unstructured data, deep neural networks are not yet a panacea for structured tabular data. In the tabular domain, their efficiency crucially relies on va…
Improving Adaptive Conformal Prediction Using Self-Supervised Learning
Nabeel Seedat, Alan Jeffares, Fergus Imrie +1
Conformal prediction is a powerful distribution-free tool for uncertainty quantification, establishing valid prediction intervals with finite-sample guarantees. To produce valid in…
Joint Training of Deep Ensembles Fails Due to Learner Collusion
Alan Jeffares, Tennison Liu, Jonathan Crabbé +1
Ensembles of machine learning models have been well established as a powerful method of improving performance over a single model. Traditionally, ensembling algorithms train their…