3.8k citations · 7.6k across the 48 of their papers we have counts for
6 papers · 1 filter
Peptide-Spectra Matching from Weak Supervision
Samuel S. Schoenholz, Sean Hackett, Laura Deming +8
As in many other scientific domains, we face a fundamental problem when using machine learning to identify proteins from mass spectrometry data: large ground truth datasets mapping…
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks
Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein +2
In recent years, state-of-the-art methods in computer vision have utilized increasingly deep convolutional neural network architectures (CNNs), with some of the most successful mod…
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
Minmin Chen, Jeffrey Pennington, Samuel S. Schoenholz
Recurrent neural networks have gained widespread use in modeling sequence data across various domains. While many successful recurrent architectures employ a notion of gating, the…
Machine learning determination of atomic dynamics at grain boundaries
Tristan A. Sharp, Spencer L. Thomas, Ekin D. Cubuk +3
In polycrystalline materials, grain boundaries are sites of enhanced atomic motion, but the complexity of the atomic structures within a grain boundary network makes it difficult t…
The Emergence of Spectral Universality in Deep Networks
Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli
Recent work has shown that tight concentration of the entire spectrum of singular values of a deep network's input-output Jacobian around one at initialization can speed up learnin…
Adversarial Spheres
Justin Gilmer, Luke Metz, Fartash Faghri +4
State of the art computer vision models have been shown to be vulnerable to small adversarial perturbations of the input. In other words, most images in the data distribution are b…