17 citations · 44 across the 16 of their papers we have counts for
9 papers · 1 filter
Wide Attention Is The Way Forward For Transformers?
Jason Ross Brown, Yiren Zhao, Ilia Shumailov +1
The Transformer is an extremely powerful and prominent deep learning architecture. In this work, we challenge the commonly held belief in deep learning that going deeper is better,…
Revisiting Structured Dropout
Yiren Zhao, Oluwatomisin Dada, Xitong Gao +1
Large neural networks are often overparameterised and prone to overfitting, Dropout is a widely used regularization technique to combat overfitting and improve model generalization…
DARTFormer: Finding The Best Type Of Attention
Jason Ross Brown, Yiren Zhao, Ilia Shumailov +1
Given the wide and ever growing range of different efficient Transformer attention mechanisms, it is important to identify which attention is most effective when given a task. In t…
ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks
Eleanor Clifford, Ilia Shumailov, Yiren Zhao +2
Early backdoor attacks against machine learning set off an arms race in attack and defence development. Defences have since appeared demonstrating some ability to detect backdoors…
Augmentation Backdoors
Joseph Rance, Yiren Zhao, Ilia Shumailov +1
Data augmentation is used extensively to improve model generalisation. However, reliance on external libraries to implement augmentation methods introduces a vulnerability into the…
Efficient Adversarial Training With Data Pruning
Maximilian Kaufmann, Yiren Zhao, Ilia Shumailov +2
Neural networks are susceptible to adversarial examples-small input perturbations that cause models to fail. Adversarial training is one of the solutions that stops adversarial exa…