672 citations · 1.5k across the 24 of their papers we have counts for
29 papers
Measuring Progress in Fine-grained Vision-and-Language Understanding
Emanuele Bugliarello, Laurent Sartran, Aishwarya Agrawal +2
While pretraining on large-scale image-text data from the Web has facilitated rapid progress on many vision-and-language (V&L) tasks, recent work has demonstrated that pretrained m…
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining
Emanuele Bugliarello, Aida Nematzadeh, Lisa Anne Hendricks
Recent work in vision-and-language pretraining has investigated supervised signals from object detection data to learn better, fine-grained multimodal representations. In this work…
Improving alignment of dialogue agents via targeted human judgements
Amelia Glaese, Nat McAleese, Maja Trębacz +31
We present Sparrow, an information-seeking dialogue agent trained to be more helpful, correct, and harmless compared to prompted language model baselines. We use reinforcement lear…
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models
Maribeth Rauh, John Mellor, Jonathan Uesato +9
Large language models produce human-like text that drive a growing number of applications. However, recent literature and, increasingly, real world observations, have demonstrated…
Training Compute-Optimal Large Language Models
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch +19
We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. We find that current large language models are si…
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Jack W. Rae, Sebastian Borgeaud, Trevor Cai +77
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.…
Ethical and social risks of harm from Language Models
Laura Weidinger, John Mellor, Maribeth Rauh +20
This paper aims to help structure the risk landscape associated with large-scale Language Models (LMs). In order to foster advances in responsible innovation, an in-depth understan…
Challenges in Detoxifying Language Models
Johannes Welbl, Amelia Glaese, Jonathan Uesato +7
Large language models (LM) generate remarkably fluent text and can be efficiently adapted across NLP tasks. Measuring and guaranteeing the quality of generated text in terms of saf…
Probing Image-Language Transformers for Verb Understanding
Lisa Anne Hendricks, Aida Nematzadeh
Multimodal image-language transformers have achieved impressive results on a variety of tasks that rely on fine-tuning (e.g., visual question answering and image retrieval). We are…
Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers
Lisa Anne Hendricks, John Mellor, Rosalia Schneider +2
Recently multimodal transformer models have gained popularity because their performance on language and vision tasks suggest they learn rich visual-linguistic representations. Focu…
Contrastive Examples for Addressing the Tyranny of the Majority
Viktoriia Sharmanska, Lisa Anne Hendricks, Trevor Darrell +1
Computer vision algorithms, e.g. for face recognition, favour groups of individuals that are better represented in the training data. This happens because of the generalization tha…
Localizing Moments in Video with Temporal Language
Lisa Anne Hendricks, Oliver Wang, Eli Shechtman +3
Localizing moments in a longer video via natural language queries is a new, challenging task at the intersection of language and video understanding. Though moment localization wit…
Object Hallucination in Image Captioning
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns +2
Despite continuously improving performance, contemporary image captioning models are prone to "hallucinating" objects that are not actually in a scene. One problem is that standard…
Grounding Visual Explanations
Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell +1
Existing visual explanation generating agents learn to fluently justify a class prediction. However, they may mention visual attributes which reflect a strong class prior, although…
Women also Snowboard: Overcoming Bias in Captioning Models (Extended Abstract)
Lisa Anne Hendricks, Kaylee Burns, Kate Saenko +2
Most machine learning methods are known to capture and exploit biases of the training data. While some biases are beneficial for learning, others are harmful. Specifically, image c…
Generating Counterfactual Explanations with Natural Language
Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell +1
Natural language explanations of deep neural network decisions provide an intuitive way for a AI agent to articulate a reasoning process. Current textual explanations learn to disc…
Women also Snowboard: Overcoming Bias in Captioning Models
Kaylee Burns, Lisa Anne Hendricks, Kate Saenko +2
Most machine learning methods are known to capture and exploit biases of the training data. While some biases are beneficial for learning, others are harmful. Specifically, image c…
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata +4
Deep models that are both effective and explainable are desirable in many settings; prior explainable models have been unimodal, offering either image-based visualization of attent…
Attentive Explanations: Justifying Decisions and Pointing to the Evidence (Extended Abstract)
Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata +4
Deep models are the defacto standard in visual decision problems due to their impressive performance on a wide array of visual tasks. On the other hand, their opaqueness has led to…
Grounding Visual Explanations (Extended Abstract)
Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell +1
Existing models which generate textual explanations enforce task relevance through a discriminative term loss function, but such mechanisms only weakly constrain mentioned object p…
Localizing Moments in Video with Natural Language
Lisa Anne Hendricks, Oliver Wang, Eli Shechtman +3
We consider retrieving a specific temporal segment, or moment, from a video given a natural language text description. Methods designed to retrieve whole video clips with natural l…
Attentive Explanations: Justifying Decisions and Pointing to the Evidence
Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata +3
Deep models are the defacto standard in visual decision models due to their impressive performance on a wide array of visual tasks. However, they are frequently seen as opaque and…
Captioning Images with Diverse Objects
Subhashini Venugopalan, Lisa Anne Hendricks, Marcus Rohrbach +3
Recent captioning models are limited in their ability to scale and describe concepts unseen in paired image-text corpora. We propose the Novel Object Captioner (NOC), a deep visual…
Improving LSTM-based Video Description with Linguistic Knowledge Mined from Text
Subhashini Venugopalan, Lisa Anne Hendricks, Raymond Mooney +1
This paper investigates how linguistic knowledge mined from large text corpora can aid the generation of natural language descriptions of videos. Specifically, we integrate both a…