450 citations · 557 across the 27 of their papers we have counts for
30 papers
On the Initialisation of Wide Low-Rank Feedforward Neural Networks
Thiziri Nait Saada, Jared Tanner
The edge-of-chaos dynamics of wide randomly initialized low-rank feedforward networks are analyzed. Formulae for the optimal weight and bias variances are extended from the full-ra…
Optimal Approximation Complexity of High-Dimensional Functions with Neural Networks
Vincent P. H. Goverse, Jad Hamdan, Jared Tanner
We investigate properties of neural networks that use both ReLU and as activation functions and build upon previous results to show that both analytic functions and functions…
Improved Projection Learning for Lower Dimensional Feature Maps
Ilan Price, Jared Tanner
The requirement to repeatedly move large feature maps off- and on-chip during inference with convolutional neural networks (CNNs) imposes high costs in terms of both energy and tim…
Tuning-free multi-coil compressed sensing MRI with Parallel Variable Density Approximate Message Passing (P-VDAMP)
Charles Millard, Mark Chiew, Jared Tanner +2
Magnetic Resonance Imaging (MRI) has excellent soft tissue contrast but is hindered by an inherently slow data acquisition process. Compressed sensing, which reconstructs sparse si…
Activation function design for deep networks: linearity and effective initialisation
Michael Murray, Vinayak Abrol, Jared Tanner
The activation function deployed in a deep neural network has great influence on the performance of the network at initialisation, which in turn has implications for training. In t…
Mutual Information of Neural Network Initialisations: Mean Field Approximations
Jared Tanner, Giuseppe Ughi
The ability to train randomly initialised deep neural networks is known to depend strongly on the variance of the weight matrices and biases as well as the choice of nonlinear acti…