102 citations · 128 across the 12 of their papers we have counts for
13 papers
Task-Agnostic Language Model Watermarking via High Entropy Passthrough Layers
Vaden Masrani, Mohammad Akbari, David Ming Xuan Yue +2
In the era of costly pre-training of large language models, ensuring the intellectual property rights of model owners, and insuring that said models are responsibly deployed, is be…
GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation
Mohsen Gholami, Mohammad Akbari, Cindy Hu +3
Knowledge distillation from LLMs is essential for the efficient deployment of language models. Prior works have proposed data generation using LLMs for preparing distilled models.…
Flexible Diffusion Modeling of Long Videos
William Harvey, Saeid Naderiparizi, Vaden Masrani +2
We present a framework for video modeling based on denoising diffusion probabilistic models that produces long-duration video completions in a variety of realistic environments. We…
Beyond Simple Meta-Learning: Multi-Purpose Models for Multi-Domain, Active and Continual Few-Shot Learning
Peyman Bateni, Jarred Barber, Raghav Goyal +4
Modern deep learning requires large-scale extensively labelled datasets for training. Few-shot learning aims to alleviate this issue by learning effectively from few labelled examp…
Proof of the impossibility of probabilistic induction
Vaden Masrani
In this short note I restate and simplify the proof of the impossibility of probabilistic induction from Popper (1992). Other proofs are possible (cf. Popper (1985)).
q-Paths: Generalizing the Geometric Annealing Path using Power Means
Vaden Masrani, Rob Brekelmans, Thang Bui +4
Many common machine learning methods involve the geometric annealing path, a sequence of intermediate densities between two distributions of interest constructed using the geometri…
Differentiable Particle Filtering without Modifying the Forward Pass
Adam Ścibior, Frank Wood
Particle filters are not compatible with automatic differentiation due to the presence of discrete resampling steps. While known estimators for the score function, based on Fisher'…
Annealed Importance Sampling with q-Paths
Rob Brekelmans, Vaden Masrani, Thang Bui +4
Annealed importance sampling (AIS) is the gold standard for estimating partition functions or marginal likelihoods, corresponding to importance sampling over a path of distribution…
Gaussian Process Bandit Optimization of the Thermodynamic Variational Objective
Vu Nguyen, Vaden Masrani, Rob Brekelmans +2
Achieving the full promise of the Thermodynamic Variational Objective (TVO), a recently proposed variational lower bound on the log evidence involving a one-dimensional Riemann int…
All in the Exponential Family: Bregman Duality in Thermodynamic Variational Inference
Rob Brekelmans, Vaden Masrani, Frank Wood +2
The recently proposed Thermodynamic Variational Objective (TVO) leverages thermodynamic integration to provide a family of variational inference objectives, which both tighten and…
Planning as Inference in Epidemiological Models
Frank Wood, Andrew Warrington, Saeid Naderiparizi +8
In this work we demonstrate how to automate parts of the infectious disease-control policy-making process via performing inference in existing epidemiological models. The kind of i…
Improved Few-Shot Visual Classification
Peyman Bateni, Raghav Goyal, Vaden Masrani +2
Few-shot learning is a fundamental task in computer vision that carries the promise of alleviating the need for exhaustively labeled data. Most few-shot learning approaches to date…
The Thermodynamic Variational Objective
Vaden Masrani, Tuan Anh Le, Frank Wood
We introduce the thermodynamic variational objective (TVO) for learning in both continuous and discrete deep generative models. The TVO arises from a key connection between variati…