498 citations · 1.6k across the 46 of their papers we have counts for
5 papers · 1 filter
Grokking at the Edge of Numerical Stability
Lucas Prieto, Melih Barsbey, Pedro A. M. Mediano +1
Grokking, the sudden generalization that occurs after prolonged overfitting, is a surprising phenomenon challenging our understanding of deep learning. Although significant progres…
From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks
Clémentine C. J. Dominé, Nicolas Anguita, Alexandra M. Proca +4
Biological and artificial neural networks develop internal representations that enable them to perform complex tasks. In artificial networks, the effectiveness of these models reli…
Learning, compression, and leakage: Minimising classification error via meta-universal compression principles
Fernando E. Rosas, Pedro A. M. Mediano, Michael Gastpar
Learning and compression are driven by the common aim of identifying and exploiting statistical regularities in data, which opens the door for fertile collaboration between these a…
Relational Forward Models for Multi-Agent Learning
Andrea Tacchetti, H. Francis Song, Pedro A. M. Mediano +5
The behavioral dynamics of multi-agent systems have a rich and orderly structure, which can be leveraged to understand these systems, and to improve how artificial agents learn to…
Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
Nat Dilokthanakul, Pedro A. M. Mediano, Marta Garnelo +4
We study a variant of the variational autoencoder model (VAE) with a Gaussian mixture as a prior distribution, with the goal of performing unsupervised clustering through deep gene…