3.8k citations · 7.4k across the 34 of their papers we have counts for
36 papers
Benchmarking Neural Network Training Algorithms
George E. Dahl, Frank Schneider, Zachary Nado +22
Training algorithms, broadly construed, are an essential part of every deep learning pipeline. Training algorithm improvements that speed up training across a wide variety of workl…
Unifying Grokking and Double Descent
Xander Davies, Lauro Langosco, David Krueger
A principled understanding of generalization in deep learning may require unifying disparate observations under a single conceptual framework. Previous work has studied \emph{grokk…
Scaling Vision Transformers to 22 Billion Parameters
Mostafa Dehghani, Josip Djolonga, Basil Mustafa +39
The scaling of Transformers has driven breakthrough capabilities for language models. At present, the largest large language models (LLMs) contain upwards of 100B parameters. Visio…
Improving Training Stability for Multitask Ranking Models in Recommender Systems
Jiaxi Tang, Yoel Drori, Daryl Chang +6
Recommender systems play an important role in many content platforms. While most recommendation research is dedicated to designing better models to improve user experience, we foun…
Leave Graphs Alone: Addressing Over-Squashing without Rewiring
Domenico Tortorella, Alessio Micheli
Recent works have investigated the role of graph bottlenecks in preventing long-range information propagation in message-passing graph neural networks, causing the so-called `over-…
A constant lower bound for the union-closed sets conjecture
Justin Gilmer
We show that for any union-closed family , there exists an which is contained in a fraction of the…