3.8k citations · 7.6k across the 48 of their papers we have counts for
6 papers · 1 filter
Self-assembling kinetics: Accessing a new design space via differentiable statistical-physics models
Carl P. Goodrich, Ella M. King, Samuel S. Schoenholz +2
The inverse problem of designing component interactions to target emergent structure is fundamental to numerous applications in biotechnology, materials science, and statistical ph…
Temperature check: theory and practice for training models with softmax-cross-entropy losses
Atish Agarwala, Jeffrey Pennington, Yann Dauphin +1
The softmax function combined with a cross-entropy loss is a principled approach to modeling probability distributions that has become ubiquitous in deep learning. The softmax func…
Unifying framework for strong and fragile liquids via machine learning: a study of liquid silica
Ekin D. Cubuk, Andrea J. Liu, Efthimios Kaxiras +1
The fragility of a glassforming liquid characterizes how rapidly its relaxation dynamics slow down with cooling. The viscosity of strong liquids follows an Arrhenius law with a tem…
Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization
Neha S. Wadia, Daniel Duckworth, Samuel S. Schoenholz +2
Machine learning is predicated on the concept of generalization: a model achieving low error on a sufficiently large training set should also perform well on novel samples from the…
Finite Versus Infinite Neural Networks: an Empirical Study
Jaehoon Lee, Samuel S. Schoenholz, Jeffrey Pennington +4
We perform a careful, thorough, and large scale empirical study of the correspondence between wide neural networks and kernel methods. By doing so, we resolve a variety of open que…
On the infinite width limit of neural networks with a standard parameterization
Jascha Sohl-Dickstein, Roman Novak, Samuel S. Schoenholz +1
There are currently two parameterizations used to derive fixed kernels corresponding to infinite width neural networks, the NTK (Neural Tangent Kernel) parameterization and the nai…