66 citations · 114 across the 17 of their papers we have counts for
18 papers
DefVerify: Do Hate Speech Models Reflect Their Dataset's Definition?
Urja Khurana, Eric Nalisnick, Antske Fokkens
When building a predictive model, it is often difficult to ensure that application-specific requirements are encoded by the model that will eventually be deployed. Consider researc…
Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks?
Urja Khurana, Eric Nalisnick, Antske Fokkens +1
Subjective tasks in NLP have been mostly relegated to objective standards, where the gold label is decided by taking the majority vote. This obfuscates annotator disagreement and t…
A Generative Model of Symmetry Transformations
James Urquhart Allingham, Bruno Kacper Mlodozeniec, Shreyas Padhy +5
Correctly capturing the symmetry transformations of data can lead to efficient models with strong generalization capabilities, though methods incorporating symmetries often require…
On the Challenges and Opportunities in Generative AI
Laura Manduchi, Clara Meister, Kushagra Pandey +23
The field of deep generative modeling has grown rapidly in the last few years. With the availability of massive amounts of training data coupled with advances in scalable unsupervi…
Beyond Top-Class Agreement: Using Divergences to Forecast Performance under Distribution Shift
Mona Schirmer, Dan Zhang, Eric Nalisnick
Knowing if a model will generalize to data 'in the wild' is crucial for safe deployment. To this end, we study model disagreement notions that consider the full predictive distribu…
One Strike, You're Out: Detecting Markush Structures in Low Signal-to-Noise Ratio Images
Thomas Jurriaans, Kinga Szarkowska, Eric Nalisnick +3
Modern research increasingly relies on automated methods to assist researchers. An example of this is Optical Chemical Structure Recognition (OCSR), which aids chemists in retrievi…
Early-Exit Neural Networks with Nested Prediction Sets
Metod Jazbec, Patrick Forré, Stephan Mandt +2
Early-exit neural networks (EENNs) enable adaptive and efficient inference by providing predictions at multiple stages during the forward pass. In safety-critical applications, the…
Active Learning for Multilingual Fingerspelling Corpora
Shuai Wang, Eric Nalisnick
We apply active learning to help with data scarcity problems in sign languages. In particular, we perform a novel analysis of the effect of pre-training. Since many sign languages…
Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity
Metod Jazbec, James Urquhart Allingham, Dan Zhang +1
Modern predictive models are often deployed to environments in which computational budgets are dynamic. Anytime algorithms are well-suited to such environments as, at any point dur…
Do Bayesian Neural Networks Need To Be Fully Stochastic?
Mrinank Sharma, Sebastian Farquhar, Eric Nalisnick +1
We investigate the benefit of treating all the parameters in a Bayesian neural network stochastically and find compelling theoretical and empirical evidence that this standard cons…
Sampling-based inference for large linear models, with application to linearised Laplace
Javier Antorán, Shreyas Padhy, Riccardo Barbano +3
Large-scale linear models are ubiquitous throughout machine learning, with contemporary application as surrogate models for neural network uncertainty quantification; that is, the…
Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitions
Urja Khurana, Ivar Vermeulen, Eric Nalisnick +2
\textbf{Offensive Content Warning}: This paper contains offensive language only for providing examples that clarify this research and do not reflect the authors' opinions. Please b…
Adapting the Linearised Laplace Model Evidence for Modern Deep Learning
Javier Antorán, David Janz, James Urquhart Allingham +4
The linearised Laplace method for estimating model uncertainty has received renewed attention in the Bayesian deep learning community. The method provides reliable error bars and a…
Adversarial Defense via Image Denoising with Chaotic Encryption
Shi Hu, Eric Nalisnick, Max Welling
In the literature on adversarial examples, white box and black box attacks have received the most attention. The adversary is assumed to have either full (white) or no (black) acce…
How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task
Urja Khurana, Eric Nalisnick, Antske Fokkens
Despite their success, modern language models are fragile. Even small changes in their training pipeline can lead to unexpected results. We study this phenomenon by examining the r…
Bayesian Deep Learning via Subnetwork Inference
Erik Daxberger, Eric Nalisnick, James Urquhart Allingham +2
The Bayesian paradigm has the potential to solve core issues of deep neural networks such as poor calibration and data inefficiency. Alas, scaling Bayesian inference to large weigh…
Predictive Complexity Priors
Eric Nalisnick, Jonathan Gordon, José Miguel Hernández-Lobato
Specifying a Bayesian prior is notoriously difficult for complex models such as neural networks. Reasoning about parameters is made challenging by the high-dimensionality and over-…
Bayesian Batch Active Learning as Sparse Subset Approximation
Robert Pinsler, Jonathan Gordon, Eric Nalisnick +1
Leveraging the wealth of unlabeled data produced in recent years provides great potential for improving supervised models. When the cost of acquiring labels is high, probabilistic…