86 citations · 179 across the 4 of their papers we have counts for
4 papers
Neural networks are a priori biased towards Boolean functions with low entropy
Chris Mingard, Joar Skalse, Guillermo Valle-Pérez +3
Understanding the inductive bias of neural networks is critical to explaining their ability to generalise. Here, for one of the simplest neural networks -- a single-layer perceptro…
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group
Mario Lezcano-Casado, David Martínez-Rubio
We introduce a novel approach to perform first-order optimization with orthogonal and unitary constraints. This approach is based on a parametrization stemming from Lie group theor…
Improvements to Inference Compilation for Probabilistic Programming in Large-Scale Scientific Simulators
Mario Lezcano Casado, Atilim Gunes Baydin, David Martinez Rubio +8
We consider the problem of Bayesian inference in the family of probabilistic models implicitly defined by stochastic generative models of data. In scientific fields ranging from po…
Online Learning Rate Adaptation with Hypergradient Descent
Atilim Gunes Baydin, Robert Cornish, David Martinez Rubio +2
We introduce a general method for improving the convergence rate of gradient-based optimizers that is easy to implement and works well in practice. We demonstrate the effectiveness…