145 citations · 188 across the 6 of their papers we have counts for
7 papers
Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team, Morgane Riviere, Shreya Pathak +195
In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In th…
Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients
Milad Alizadeh, Shyam A. Tailor, Luisa M Zintgraf +4
Pruning neural networks at initialization would enable us to find sparse models that retain the accuracy of the original network while consuming fewer computational resources for t…
Deep Deterministic Uncertainty for Semantic Segmentation
Jishnu Mukhoti, Joost van Amersfoort, Philip H. S. Torr +1
We extend Deep Deterministic Uncertainty (DDU), a method for uncertainty estimation using feature space densities, to semantic segmentation. DDU enables quantifying and disentangli…
Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective
Lewis Smith, Joost van Amersfoort, Haiwen Huang +2
ResNets constrained to be bi-Lipschitz, that is, approximately distance preserving, have been a crucial component of recently proposed techniques for deterministic uncertainty quan…
Single Shot Structured Pruning Before Training
Joost van Amersfoort, Milad Alizadeh, Sebastian Farquhar +2
We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training. Unlike previous works on pruning befo…
BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
Andreas Kirsch, Joost van Amersfoort, Yarin Gal
We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple…