57 citations · 79 across the 14 of their papers we have counts for
16 papers
Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM
Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal +2
Contrastive decoding (CD) (Li et al., 2023) improves the next-token distribution of a large expert language model (LM) using a small amateur LM. Although CD is applied to various L…
Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy
Alberto Testoni, Raffaella Bernardi
Generating goal-oriented questions in Visual Dialogue tasks is a challenging and long-standing problem. State-Of-The-Art systems are shown to generate questions that, although gram…
Overprotective Training Environments Fall Short at Testing Time: Let Models Contribute to Their Own Training
Alberto Testoni, Raffaella Bernardi
Despite important progress, conversational systems often generate dialogues that sound unnatural to humans. We conjecture that the reason lies in their different training and testi…
The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues
Alberto Testoni, Raffaella Bernardi
When training a model on referential dialogue guessing games, the best model is usually chosen based on its task success. We show that in the popular end-to-end approach, this choi…
Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering
Claudio Greco, Barbara Plank, Raquel Fernández +1
We study the issue of catastrophic forgetting in the context of neural multimodal approaches to Visual Question Answering (VQA). Motivated by evidence from psycholinguistics, we de…
Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat
Ravi Shekhar, Aashish Venkatesh, Tim Baumgärtner +4
We propose a grounded dialogue state encoder which addresses a foundational issue on how to integrate visual grounding with dialogue system components. As a test-bed, we focus on t…
Grounded Textual Entailment
Hoa Trong Vu, Claudio Greco, Aliia Erofeeva +6
Capturing semantic relations between sentences, such as entailment, is a long-standing challenge for computational semantics. Logic-based models analyse entailment in terms of poss…
Ask No More: Deciding when to guess in referential visual dialogue
Ravi Shekhar, Tim Baumgartner, Aashish Venkatesh +3
Our goal is to explore how the abilities brought in by a dialogue manager can be included in end-to-end visually grounded conversational agents. We make initial steps towards this…
Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in Predicting Quantifiers
Sandro Pezzelle, Shane Steinert-Threlkeld, Raffaela Bernardi +1
We study the role of linguistic context in predicting quantifiers (`few', `all'). We collect crowdsourced data from human participants and test various models in a local (single-se…
Comparatives, Quantifiers, Proportions: A Multi-Task Model for the Learning of Quantities from Vision
Sandro Pezzelle, Ionut-Teodor Sorodoc, Raffaella Bernardi
The present work investigates whether different quantification mechanisms (set comparison, vague quantification, and proportional estimation) can be jointly learned from visual sce…
FOIL it! Find One mismatch between Image and Language caption
Ravi Shekhar, Sandro Pezzelle, Yauhen Klimovich +4
In this paper, we aim to understand whether current language and vision (LaVi) models truly grasp the interaction between the two modalities. To this end, we propose an extension o…
Pay Attention to Those Sets! Learning Quantification from Images
Ionut Sorodoc, Sandro Pezzelle, Aurélie Herbelot +2
Major advances have recently been made in merging language and vision representations. But most tasks considered so far have confined themselves to the processing of objects and le…
Be Precise or Fuzzy: Learning the Meaning of Cardinals and Quantifiers from Vision
Sandro Pezzelle, Marco Marelli, Raffaella Bernardi
People can refer to quantities in a visual scene by using either exact cardinals (e.g. one, two, three) or natural language quantifiers (e.g. few, most, all). In humans, these two…
The LAMBADA dataset: Word prediction requiring a broad discourse context
Denis Paperno, Germán Kruszewski, Angeliki Lazaridou +6
We introduce LAMBADA, a dataset to evaluate the capabilities of computational models for text understanding by means of a word prediction task. LAMBADA is a collection of narrative…
Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
Raffaella Bernardi, Ruket Cakici, Desmond Elliott +6
Automatic description generation from natural images is a challenging problem that has recently received a large amount of interest from the computer vision and natural language pr…
Unveiling the Dreams of Word Embeddings: Towards Language-Driven Image Generation
Angeliki Lazaridou, Dat Tien Nguyen, Raffaella Bernardi +1
We introduce language-driven image generation, the task of generating an image visualizing the semantic contents of a word embedding, e.g., given the word embedding of grasshopper,…