637 citations · 1.9k across the 20 of their papers we have counts for
21 papers
Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures
Emmanuel Abbe, Samy Bengio, Elisabetta Cornacchia +4
This paper considers the Pointer Value Retrieval (PVR) benchmark introduced in [ZRKB21], where a 'reasoning' function acts on a string of digits to produce the label. More generall…
On the Origins of the Block Structure Phenomenon in Neural Network Representations
Thao Nguyen, Maithra Raghu, Simon Kornblith
Recent work has uncovered a striking phenomenon in large-capacity neural networks: they contain blocks of contiguous hidden layers with highly similar representations. This block s…
Do Vision Transformers See Like Convolutional Neural Networks?
Maithra Raghu, Thomas Unterthiner, Simon Kornblith +2
Convolutional neural networks (CNNs) have so far been the de-facto model for visual data. Recent work has shown that (Vision) Transformer models (ViT) can achieve comparable or eve…
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization
Chiyuan Zhang, Maithra Raghu, Jon Kleinberg +1
Central to the success of artificial neural networks is their ability to generalize. But does neural network generalization primarily rely on seeing highly similar training example…
Teaching with Commentaries
Aniruddh Raghu, Maithra Raghu, Simon Kornblith +2
Effective training of deep neural networks can be challenging, and there remain many open questions on how to best learn these models. Recently developed methods to improve neural…
Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth
Thao Nguyen, Maithra Raghu, Simon Kornblith
A key factor in the success of deep neural networks is the ability to scale models to improve performance by varying the architecture depth and width. This simple property of neura…