55 citations · 82 across the 12 of their papers we have counts for
12 papers
Language Re-generation: An investigation into information locality effects on reconstruction
Amirhossein Mohammadi, Laurence E. Frank, Albert Gatt +1
Information locality, the tendency for syntactically related words to appear close together, shapes both human language processing and language model learning. While prior work has…
Contrast Is All You Need
Burak Kilic, Florix Bex, Albert Gatt
In this study, we analyze data-scarce classification scenarios, where available labeled legal data is small and imbalanced, potentially hurting the quality of the results. We focus…
Interpreting Vision and Language Generative Models with Semantic Visual Priors
Michele Cafagna, Lina M. Rojas-Barahona, Kees van Deemter +1
When applied to Image-to-text models, interpretability methods often provide token-by-token explanations namely, they compute a visual explanation for each token of the generated s…
HL Dataset: Visually-grounded Description of Scenes, Actions and Rationales
Michele Cafagna, Kees van Deemter, Albert Gatt
Current captioning datasets focus on object-centric captions, describing the visible objects in the image, e.g. "people eating food in a park". Although these datasets are useful t…
Understanding Cross-modal Interactions in V&L Models that Generate Scene Descriptions
Michele Cafagna, Kees van Deemter, Albert Gatt
Image captioning models tend to describe images in an object-centric way, emphasising visible objects. But image descriptions can also abstract away from objects and describe the t…
Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese
Kurt Micallef, Albert Gatt, Marc Tanti +2
Multilingual language models such as mBERT have seen impressive cross-lingual transfer to a variety of languages, but many languages remain excluded from these models. In this pape…
Face2Text revisited: Improved data set and baseline results
Marc Tanti, Shaun Abdilla, Adrian Muscat +3
Current image description generation models do not transfer well to the task of describing human faces. To encourage the development of more human-focused descriptions, we develope…
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena
Letitia Parcalabescu, Michele Cafagna, Lilitta Muradjan +3
We propose VALSE (Vision And Language Structured Evaluation), a novel benchmark designed for testing general-purpose pretrained vision and language (V&L) models for their visio-lin…
Analysis of Data Augmentation Methods for Low-Resource Maltese ASR
Andrea DeMarco, Carlos Mena, Albert Gatt +3
Recent years have seen an increased interest in the computational speech processing of Maltese, but resources remain sparse. In this paper, we consider data augmentation techniques…
What Vision-Language Models `See' when they See Scenes
Michele Cafagna, Kees van Deemter, Albert Gatt
Images can be described in terms of the objects they contain, or in terms of the types of scene or place that they instantiate. In this paper we address to what extent pretrained V…
On the Language-specificity of Multilingual BERT and the Impact of Fine-tuning
Marc Tanti, Lonneke van der Plas, Claudia Borg +1
Recent work has shown evidence that the knowledge acquired by multilingual BERT (mBERT) has two components: a language-specific and a language-neutral one. This paper analyses the…
On Architectures for Including Visual Information in Neural Language Models for Image Description
Marc Tanti, Albert Gatt, Kenneth P. Camilleri
A neural language model can be conditioned into generating descriptions for images by providing visual information apart from the sentence prefix. This visual information can be in…