258 citations · 952 across the 9 of their papers we have counts for
13 papers
Exploring Low Rank Training of Deep Neural Networks
Siddhartha Rao Kamalakara, Acyr Locatelli, Bharat Venkitesh +3
Training deep neural networks in low rank, i.e. with factorised layers, is of particular interest to the community: it offers efficiency over unfactorised training in terms of both…
Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval
Pascal Notin, Mafalda Dias, Jonathan Frazer +4
The ability to accurately model the fitness landscape of protein sequences is critical to a wide range of applications, from quantifying the effects of human variants on disease li…
Robustness to Pruning Predicts Generalization in Deep Neural Networks
Lorenz Kuhn, Clare Lyle, Aidan N. Gomez +2
Existing generalization measures that aim to capture a model's simplicity based on parameter counts or norms fail to explain generalization in overparameterized deep neural network…
Improving compute efficacy frontiers with SliceOut
Pascal Notin, Aidan N. Gomez, Joanna Yoo +1
Pushing forward the compute efficacy frontier in deep learning is critical for tasks that require frequent model re-training or workloads that entail training a large number of mod…
Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers
Tim Z. Xiao, Aidan N. Gomez, Yarin Gal
We detect out-of-training-distribution sentences in Neural Machine Translation using the Bayesian Deep Learning equivalent of Transformer models. For this we develop a new measure…
A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks
Angelos Filos, Sebastian Farquhar, Aidan N. Gomez +6
Evaluation of Bayesian deep learning (BDL) methods is challenging. We often seek to evaluate the methods' robustness and scalability, assessing whether new tools give `better' unce…