25 citations · 85 across the 21 of their papers we have counts for
23 papers
Dynamic Classifier-Free Diffusion Guidance via Online Feedback
Pinelopi Papalampidi, Olivia Wiles, Ira Ktena +5
Classifier-free guidance (CFG) is a cornerstone of text-to-image diffusion models, yet its effectiveness is limited by the use of static guidance scales. This "one-size-fits-all" a…
MuLan: A Study of Fact Mutability in Language Models
Constanza Fierro, Nicolas Garneau, Emanuele Bugliarello +2
Facts are subject to contingencies and can be true or false in different circumstances. One such contingency is time, wherein some facts mutate over a given period, e.g., the presi…
Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings
Olivia Wiles, Chuhan Zhang, Isabela Albuquerque +11
While text-to-image (T2I) generative models have become ubiquitous, they do not necessarily generate images that align with a given prompt. While previous work has evaluated T2I al…
On the Interplay between Fairness and Explainability
Stephanie Brandl, Emanuele Bugliarello, Ilias Chalkidis
In order to build reliable and trustworthy NLP applications, models need to be both fair across different demographics and explainable. Usually these two objectives, fairness and e…
StoryBench: A Multifaceted Benchmark for Continuous Story Visualization
Emanuele Bugliarello, Hernan Moraldo, Ruben Villegas +7
Generating video stories from text prompts is a complex task. In addition to having high visual quality, videos need to realistically adhere to a sequence of text prompts whilst be…
Measuring Progress in Fine-grained Vision-and-Language Understanding
Emanuele Bugliarello, Laurent Sartran, Aishwarya Agrawal +2
While pretraining on large-scale image-text data from the Web has facilitated rapid progress on many vision-and-language (V&L) tasks, recent work has demonstrated that pretrained m…
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining
Emanuele Bugliarello, Aida Nematzadeh, Lisa Anne Hendricks
Recent work in vision-and-language pretraining has investigated supervised signals from object detection data to learn better, fine-grained multimodal representations. In this work…
A Study of Autoregressive Decoders for Multi-Tasking in Computer Vision
Lucas Beyer, Bo Wan, Gagan Madan +9
There has been a recent explosion of computer vision models which perform many tasks and are composed of an image encoder (usually a ViT) and an autoregressive decoder (usually a T…
Multilingual Multimodal Learning with Machine Translated Text
Chen Qiu, Dan Oneata, Emanuele Bugliarello +2
Most vision-and-language pretraining research focuses on English tasks. However, the creation of multilingual multimodal evaluation datasets (e.g. Multi30K, xGQA, XVNLI, and MaRVL)…
Language Modelling with Pixels
Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello +3
Language models are defined over a finite set of inputs, which creates a vocabulary bottleneck when we attempt to scale the number of supported languages. Tackling this bottleneck…
Ancestor-to-Creole Transfer is Not a Walk in the Park
Heather Lent, Emanuele Bugliarello, Anders Søgaard
We aim to learn language models for Creole languages for which large volumes of data are not readily available, and therefore explore the potential transfer from ancestor languages…
Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization
Aishwarya Agrawal, Ivana Kajić, Emanuele Bugliarello +4
Vision-and-language (V&L) models pretrained on large-scale multimodal data have demonstrated strong performance on various tasks such as image captioning and visual question answer…
Mostra: A Flexible Balancing Framework to Trade-off User, Artist and Platform Objectives for Music Sequencing
Emanuele Bugliarello, Rishabh Mehrotra, James Kirk +1
We consider the task of sequencing tracks on music streaming platforms where the goal is to maximise not only user satisfaction, but also artist- and platform-centric objectives, n…
Challenges and Strategies in Cross-Cultural NLP
Daniel Hershcovich, Stella Frank, Heather Lent +11
Various efforts in the Natural Language Processing (NLP) community have been made to accommodate linguistic diversity and serve speakers of many different languages. However, it is…
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages
Emanuele Bugliarello, Fangyu Liu, Jonas Pfeiffer +4
Reliable evaluation benchmarks designed for replicability and comprehensiveness have driven progress in machine learning. Due to the lack of a multilingual benchmark, however, visi…
Visually Grounded Reasoning across Languages and Cultures
Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti +3
The design of widespread vision-and-language datasets and pre-trained encoders directly adopts, or draws inspiration from, the concepts and images of ImageNet. While one can hardly…
On Language Models for Creoles
Heather Lent, Emanuele Bugliarello, Miryam de Lhoneux +2
Creole languages such as Nigerian Pidgin English and Haitian Creole are under-resourced and largely ignored in the NLP literature. Creoles typically result from the fusion of a for…
Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers
Stella Frank, Emanuele Bugliarello, Desmond Elliott
Pretrained vision-and-language BERTs aim to learn representations that combine information from both modalities. We propose a diagnostic method based on cross-modal input ablation…
The Role of Syntactic Planning in Compositional Image Captioning
Emanuele Bugliarello, Desmond Elliott
Image captioning has focused on generalizing to images drawn from the same distribution as the training set, and not to the more challenging problem of generalizing to different di…
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs
Emanuele Bugliarello, Ryan Cotterell, Naoaki Okazaki +1
Large-scale pretraining and task-specific fine-tuning is now the standard methodology for many tasks in computer vision and natural language processing. Recently, a multitude of me…
It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information
Emanuele Bugliarello, Sabrina J. Mielke, Antonios Anastasopoulos +2
The performance of neural machine translation systems is commonly evaluated in terms of BLEU. However, due to its reliance on target language properties and generation, the BLEU me…
Enhancing Machine Translation with Dependency-Aware Self-Attention
Emanuele Bugliarello, Naoaki Okazaki
Most neural machine translation models only rely on pairs of parallel sentences, assuming syntactic information is automatically learned by an attention mechanism. In this work, we…
Matrix Completion in the Unit Hypercube via Structured Matrix Factorization
Emanuele Bugliarello, Swayambhoo Jain, Vineeth Rakesh
Several complex tasks that arise in organizations can be simplified by mapping them into a matrix completion problem. In this paper, we address a key challenge faced by our company…