3 citations · 4 across the 3 of their papers we have counts for
8 papers
Optimisation & Generalisation in Networks of Neurons
Jeremy Bernstein
The goal of this thesis is to develop the optimisation and generalisation theoretic foundations of learning in artificial neural networks. On optimisation, a new theoretical framew…
Fine-Grained System Identification of Nonlinear Neural Circuits
Dawna Bagherian, James Gornet, Jeremy Bernstein +3
We study the problem of sparse nonlinear model recovery of high dimensional compositional functions. Our study is motivated by emerging opportunities in neuroscience to recover fin…
Computing the Information Content of Trained Neural Networks
Jeremy Bernstein, Yisong Yue
How much information does a learning algorithm extract from the training data and store in a neural network's weights? Too much, and the network would overfit to the training data.…
Learning by Turning: Neural Architecture Aware Optimisation
Yang Liu, Jeremy Bernstein, Markus Meister +1
Descent methods for deep networks are notoriously capricious: they require careful tuning of step size, momentum and weight decay, and which method will work best on a new benchmar…
Learning compositional functions via multiplicative weight updates
Jeremy Bernstein, Jiawei Zhao, Markus Meister +3
Compositionality is a basic structural feature of both biological and artificial neural networks. Learning compositional functions via gradient descent incurs well known problems l…
On the distance between two neural networks and the stability of learning
Jeremy Bernstein, Arash Vahdat, Yisong Yue +1
This paper relates parameter distance to gradient breakdown for a broad class of nonlinear compositional functions. The analysis leads to a new distance function called deep relati…