22 citations · 92 across the 18 of their papers we have counts for
25 papers
Uncertainty in Natural Language Generation: From Theory to Applications
Joris Baan, Nico Daheim, Evgenia Ilia +7
Recent advances of powerful Language Models have allowed Natural Language Generation (NLG) to emerge as an important technology that can not only perform traditional tasks like sum…
What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability
Mario Giulianelli, Joris Baan, Wilker Aziz +2
In Natural Language Generation (NLG) tasks, for any input, multiple communicative goals are plausible, and any goal can be put into words, or produced, in multiple ways. We charact…
VISION DIFFMASK: Faithful Interpretation of Vision Transformers with Differentiable Patch Masking
Angelos Nalmpantis, Apostolos Panagiotopoulos, John Gkountouras +2
The lack of interpretability of the Vision Transformer may hinder its use in critical real-world applications despite its effectiveness. To overcome this issue, we propose a post-h…
Stop Measuring Calibration When Humans Disagree
Joris Baan, Wilker Aziz, Barbara Plank +1
Calibration is a popular framework to evaluate whether a classifier knows when it does not know - i.e., its predictive probabilities are a good indication of how likely a predictio…
Statistical Model Criticism of Variational Auto-Encoders
Claartje Barkhof, Wilker Aziz
We propose a framework for the statistical evaluation of variational auto-encoders (VAEs) and test two instances of this framework in the context of modelling images of handwritten…
Highly Parallel Autoregressive Entity Linking with Discriminative Correction
Nicola De Cao, Wilker Aziz, Ivan Titov
Generative approaches have been recently shown to be effective for both Entity Disambiguation and Entity Linking (i.e., joint mention detection and disambiguation). However, the pr…
Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation
Bryan Eikema, Wilker Aziz
In NMT we search for the mode of the model distribution to form predictions. The mode and other high-probability translations found by beam search have been shown to often be inade…
Sparse Communication via Mixed Distributions
António Farinhas, Wilker Aziz, Vlad Niculae +1
Neural networks and other machine learning models compute continuous representations, while humans communicate mostly through discrete symbols. Reconciling these two forms of commu…
Editing Factual Knowledge in Language Models
Nicola De Cao, Wilker Aziz, Ivan Titov
The factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual infe…
Disease Normalization with Graph Embeddings
Dhruba Pujary, Camilo Thorne, Wilker Aziz
The detection and normalization of diseases in biomedical texts are key biomedical natural language processing tasks. Disease names need not only be identified, but also normalized…
Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity
Gonçalo M. Correia, Vlad Niculae, Wilker Aziz +1
Training neural network models with discrete (categorical or structured) latent variables can be computationally challenging, due to the need for marginalization over large or comb…
The Power Spherical distribution
Nicola De Cao, Wilker Aziz
There is a growing interest in probabilistic models defined in hyper-spherical spaces, be it to accommodate observed data or latent structure. The von Mises-Fisher (vMF) distributi…
Latent Transformations for Discrete-Data Normalising Flows
Rob Hesselink, Wilker Aziz
Normalising flows (NFs) for discrete data are challenging because parameterising bijective transformations of discrete variables requires predicting discrete/integer parameters. Ha…
Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation
Bryan Eikema, Wilker Aziz
Recent studies have revealed a number of pathologies of neural machine translation (NMT) systems. Hypotheses explaining these mostly suggest there is something fundamentally wrong…
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking
Nicola De Cao, Michael Schlichtkrull, Wilker Aziz +1
Attribution methods assess the contribution of inputs to the model prediction. One way to do so is erasure: a subset of inputs is considered irrelevant if it can be removed without…
A Latent Morphology Model for Open-Vocabulary Neural Machine Translation
Duygu Ataman, Wilker Aziz, Alexandra Birch
Translation into morphologically-rich languages challenges neural machine translation (NMT) models with extremely sparse vocabularies where atomic treatment of surface forms is unr…
Block Neural Autoregressive Flow
Nicola De Cao, Ivan Titov, Wilker Aziz
Normalising flows (NFS) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to hand-crafted…
Effective Estimation of Deep Generative Language Models
Tom Pelsmaeker, Wilker Aziz
Advances in variational inference enable parameterisation of probabilistic models by deep neural networks. This combines the statistical transparency of the probabilistic modelling…
Modeling Latent Sentence Structure in Neural Machine Translation
Jasmijn Bastings, Wilker Aziz, Ivan Titov +1
Recently it was shown that linguistic structure predicted by a supervised parser can be beneficial for neural machine translation (NMT). In this work we investigate a more challeng…
Latent Variable Model for Multi-modal Translation
Iacer Calixto, Miguel Rios, Wilker Aziz
In this work, we propose to model the interaction between visual and textual features for multi-modal neural machine translation (MMT) through a latent variable model. This latent…
Auto-Encoding Variational Neural Machine Translation
Bryan Eikema, Wilker Aziz
We present a deep generative model of bilingual sentence pairs for machine translation. The model generates source and target sentences jointly from a shared latent representation…
A Stochastic Decoder for Neural Machine Translation
Philip Schulz, Wilker Aziz, Trevor Cohn
The process of translation is ambiguous, in that there are typically many valid trans- lations for a given sentence. This gives rise to significant variation in parallel cor- pora,…
Deep Generative Model for Joint Alignment and Word Representation
Miguel Rios, Wilker Aziz, Khalil Sima'an
This work exploits translation data as a source of semantically relevant learning signal for models of word representation. In particular, we exploit equivalence through translatio…
Graph Convolutional Encoders for Syntax-aware Neural Machine Translation
Jasmijn Bastings, Ivan Titov, Wilker Aziz +2
We present a simple and effective approach to incorporating syntactic structure into neural attention-based encoder-decoder models for machine translation. We rely on graph-convolu…