46 citations · 76 across the 5 of their papers we have counts for
5 papers
Unifying Grokking and Double Descent
Xander Davies, Lauro Langosco, David Krueger
A principled understanding of generalization in deep learning may require unifying disparate observations under a single conceptual framework. Previous work has studied \emph{grokk…
Leave Graphs Alone: Addressing Over-Squashing without Rewiring
Domenico Tortorella, Alessio Micheli
Recent works have investigated the role of graph bottlenecks in preventing long-range information propagation in message-passing graph neural networks, causing the so-called `over-…
Normalizing flows for atomic solids
Peter Wirnsberger, George Papamakarios, Borja Ibarz +4
We present a machine-learning approach, based on normalizing flows, for modelling atomic solids. Our model transforms an analytically tractable base distribution into the target so…
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping
James Martens, Andy Ballard, Guillaume Desjardins +4
Using an extended and formalized version of the Q/C map analysis of Poole et al. (2016), along with Neural Tangent Kernel theory, we identify the main pathologies present in deep n…
Unsupervised Doodling and Painting with Improved SPIRAL
John F. J. Mellor, Eunbyung Park, Yaroslav Ganin +7
We investigate using reinforcement learning agents as generative models of images (extending arXiv:1804.01118). A generative agent controls a simulated painting environment, and is…