3.8k citations · 6.1k across the 26 of their papers we have counts for
8 papers · 1 filter
Measuring the Effects of Data Parallelism on Neural Network Training
Christopher J. Shallue, Jaehoon Lee, Joseph Antognini +3
Recent hardware developments have dramatically increased the scale of data parallelism available for neural network training. Among the simplest ways to harness next-generation har…
The Importance of Generation Order in Language Modeling
Nicolas Ford, Daniel Duckworth, Mohammad Norouzi +1
Neural language models are a critical component of state-of-the-art systems for machine translation, summarization, audio transcription, and other tasks. These language models are…
Peptide-Spectra Matching from Weak Supervision
Samuel S. Schoenholz, Sean Hackett, Laura Deming +8
As in many other scientific domains, we face a fundamental problem when using machine learning to identify proteins from mass spectrometry data: large ground truth datasets mapping…
Motivating the Rules of the Game for Adversarial Example Research
Justin Gilmer, Ryan P. Adams, Ian Goodfellow +2
Advances in machine learning have led to broad deployment of systems with impressive performance on important problems. Nonetheless, these systems can be induced to make errors on…
Embedding Text in Hyperbolic Spaces
Bhuwan Dhingra, Christopher J. Shallue, Mohammad Norouzi +2
Natural language text exhibits hierarchical structure in a variety of respects. Ideally, we could incorporate our prior knowledge of this hierarchical structure into unsupervised l…
Relational inductive biases, deep learning, and graph networks
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst +24
Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, i…