7 citations · 11 across the 5 of their papers we have counts for
3 papers · 1 filter
NRGPT: An Energy-based Alternative for GPT
Nima Dehmamy, Benjamin Hoover, Bishwajit Saha +3
Generative Pre-trained Transformer (GPT) architectures are the most popular design for language modeling. Energy-based modeling is a different paradigm that views inference as a dy…
Contraction Properties of the Global Workspace Primitive
Michaela Ennis, Leo Kozachkov, Jean-Jacques Slotine
To push forward the important emerging research field surrounding multi-area recurrent neural networks (RNNs), we expand theoretically and empirically on the provably stable RNNs o…
Generalization in Supervised Learning Through Riemannian Contraction
Leo Kozachkov, Patrick M. Wensing, Jean-Jacques Slotine
We prove that Riemannian contraction in a supervised learning setting implies generalization. Specifically, we show that if an optimizer is contracting in some Riemannian metric wi…