297 citations · 1.1k across the 15 of their papers we have counts for
15 papers
Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team, Morgane Riviere, Shreya Pathak +195
In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In th…
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Petko Georgiev, Ving Ian Lei +1132
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over…
Informative Priors Improve the Reliability of Multimodal Clinical Data Classification
L. Julian Lechuga Lopez, Tim G. J. Rudner, Farah E. Shamout
Machine learning-aided clinical decision support has the potential to significantly improve patient care. However, existing efforts in this domain for principled quantification of…
Plex: Towards Reliability using Pretrained Large Model Extensions
Dustin Tran, Jeremiah Liu, Michael W. Dusenberry +23
A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures. P…
Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients
Milad Alizadeh, Shyam A. Tailor, Luisa M Zintgraf +4
Pruning neural networks at initialization would enable us to find sparse models that retain the accuracy of the original network while consuming fewer computational resources for t…
Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data
Andrew Jesson, Panagiotis Tigas, Joost van Amersfoort +3
Estimating personalized treatment effects from high-dimensional observational data is essential in situations where experimental designs are infeasible, unethical, or expensive. Ex…
Deep Deterministic Uncertainty for Semantic Segmentation
Jishnu Mukhoti, Joost van Amersfoort, Philip H. S. Torr +1
We extend Deep Deterministic Uncertainty (DDU), a method for uncertainty estimation using feature space densities, to semantic segmentation. DDU enables quantifying and disentangli…
Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective
Lewis Smith, Joost van Amersfoort, Haiwen Huang +2
ResNets constrained to be bi-Lipschitz, that is, approximately distance preserving, have been a crucial component of recently proposed techniques for deterministic uncertainty quan…
Deep Deterministic Uncertainty: A Simple Baseline
Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort +2
Reliable uncertainty from deterministic single-forward pass models is sought after because conventional methods of uncertainty quantification are computationally expensive. We take…
Single Shot Structured Pruning Before Training
Joost van Amersfoort, Milad Alizadeh, Sebastian Farquhar +2
We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training. Unlike previous works on pruning befo…
BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
Andreas Kirsch, Joost van Amersfoort, Yarin Gal
We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple…
Deep Hashing using Entropy Regularised Product Quantisation Network
Jo Schlemper, Jose Caballero, Andy Aitken +1
In large scale systems, approximate nearest neighbour search is a crucial algorithm to enable efficient data retrievals. Recently, deep learning-based hashing algorithms have been…
Frame Interpolation with Multi-Scale Deep Loss Functions and Generative Adversarial Networks
Joost van Amersfoort, Wenzhe Shi, Alejandro Acosta +4
Frame interpolation attempts to synthesise frames given one or more consecutive video frames. In recent years, deep learning approaches, and notably convolutional neural networks,…
Transformation-Based Models of Video Sequences
Joost van Amersfoort, Anitha Kannan, Marc'Aurelio Ranzato +3
In this work we propose a simple unsupervised approach for next frame prediction in video. Instead of directly predicting the pixels in a frame given past frames, we predict the tr…
Variational Recurrent Auto-Encoders
Otto Fabius, Joost R. van Amersfoort
In this paper we propose a model that combines the strengths of RNNs and SGVB: the Variational Recurrent Auto-Encoder (VRAE). Such a model can be used for efficient, large scale un…