Publications (41)
Activation function design for deep networks: linearity and effective initialisation
Michael Murray, Vinayak Abrol, Jared Tanner
The activation function deployed in a deep neural network has great influence on the performance of the network at initialisation, which in turn has implications for training. In t…
Compressed Sensing: How sharp is the Restricted Isometry Property
Jeffrey D. Blanchard, Coralia Cartis, Jared Tanner
Compressed Sensing (CS) seeks to recover an unknown vector with entries by making far fewer than measurements; it posits that the number of compressed sensing measurements…
Theory of Minimal Weight Perturbations in Deep Networks and its Applications for Low-Rank Activated Backdoor Attacks
Bethan Evans, Jared Tanner
The minimal norm weight perturbations of DNNs required to achieve a specified change in output are derived and the factors determining its size are discussed. These single-layer ex…
A robust parallel algorithm for combinatorial compressed sensing
Rodrigo Mendoza-Smith, Jared Tanner, Florian Wechsung
In previous work two of the authors have shown that a vector with at most nonzeros can be recovered from an expander sketch in $\mathcal{O}(\mathr…
Encoder blind combinatorial compressed sensing
Michael Murray, Jared Tanner
In its most elementary form, compressed sensing studies the design of decoding algorithms to recover a sufficiently sparse vector or code from a lower dimensional linear measuremen…
Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers
Thiziri Nait Saada, Alireza Naderi, Jared Tanner
Attention layers are the core component of transformers, the current state-of-the-art neural network architecture. Alternatives to softmax-based attention are being explored due to…