3.8k citations · 7.6k across the 48 of their papers we have counts for
8 papers · 1 filter
Mean Field Residual Networks: On the Edge of Chaos
Greg Yang, Samuel S. Schoenholz
We study randomly initialized residual networks using mean field theory and the theory of difference equations. Classical feedforward neural networks, such as those with tanh activ…
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli
It is well known that the initialization of weights in deep neural networks can have a dramatic impact on learning speed. For example, ensuring the mean squared singular value of a…
Intriguing Properties of Adversarial Examples
Ekin D. Cubuk, Barret Zoph, Samuel S. Schoenholz +1
It is becoming increasingly clear that many machine learning classifiers are vulnerable to adversarial examples. In attempting to explain the origin of adversarial examples, previo…
Deep Neural Networks as Gaussian Processes
Jaehoon Lee, Yasaman Bahri, Roman Novak +3
It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinit…
A Correspondence Between Random Neural Networks and Statistical Field Theory
Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein
A number of recent papers have provided evidence that practical design questions about neural networks may be tackled theoretically by studying the behavior of random networks. How…
Combining Machine Learning and Physics to Understand Glassy Systems
Samuel S. Schoenholz
Our understanding of supercooled liquids and glasses has lagged significantly behind that of simple liquids and crystalline solids. This is in part due to the many possibly relevan…
Neural Message Passing for Quantum Chemistry
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley +2
Supervised learning on molecules has incredible potential to be useful in chemistry, drug discovery, and materials science. Luckily, several promising and closely related neural ne…
Machine learning prediction errors better than DFT accuracy
Felix A. Faber, Luke Hutchison, Bing Huang +7
We investigate the impact of choosing regressors and molecular representations for the construction of fast machine learning (ML) models of thirteen electronic ground-state propert…