10 citations · 26 across the 16 of their papers we have counts for
18 papers
Dynamic Planning with a LLM
Gautier Dagan, Frank Keller, Alex Lascarides
While Large Language Models (LLMs) can solve many NLP tasks in zero-shot settings, applications involving embodied agents remain problematic. In particular, complex plans that requ…
Meta-learning For Vision-and-language Cross-lingual Transfer
Hanxu Hu, Frank Keller
Current pre-trained vison-language models (PVLMs) achieve excellent performance on a range of multi-modal datasets. Recent work has aimed at building multilingual models, and a ran…
Detecting and Grounding Important Characters in Visual Stories
Danyang Liu, Frank Keller
Characters are essential to the plot of any story. Establishing the characters before writing a story can improve the clarity of the plot and the overall flow of the narrative. How…
Learning Action Changes by Measuring Verb-Adverb Textual Relationships
Davide Moltisanti, Frank Keller, Hakan Bilen +1
The goal of this work is to understand the way actions are performed in videos. That is, given a video, we aim to predict an adverb indicating a modification applied to the action…
Learning the Effects of Physical Actions in a Multi-modal Environment
Gautier Dagan, Frank Keller, Alex Lascarides
Large Language Models (LLMs) handle physical commonsense information inadequately. As a result of being trained in a disembodied setting, LLMs often fail to predict an action's out…
Who are you referring to? Coreference resolution in image narrations
Arushi Goel, Basura Fernando, Frank Keller +1
Coreference resolution aims to identify words and phrases which refer to same entity in a text, a core task in natural language processing. In this paper, we extend this task to re…
A Closer Look at Temporal Ordering in the Segmentation of Instructional Videos
Anil Batra, Shreyank N Gowda, Frank Keller +1
Understanding the steps required to perform a task is an important skill for AI systems. Learning these steps from instructional videos involves two subproblems: (i) identifying th…
Learn2Augment: Learning to Composite Videos for Data Augmentation in Action Recognition
Shreyank N Gowda, Marcus Rohrbach, Frank Keller +1
We address the problem of data augmentation for video action recognition. Standard augmentation strategies in video are hand-designed and sample the space of possible augmented dat…
Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation
Arushi Goel, Basura Fernando, Frank Keller +1
Scene graph generation (SGG) aims to capture a wide variety of interactions between pairs of objects, which is essential for full scene understanding. Existing SGG methods trained…
Finding the Right Moment: Human-Assisted Trailer Creation via Task Composition
Pinelopi Papalampidi, Frank Keller, Mirella Lapata
Movie trailers perform multiple functions: they introduce viewers to the story, convey the mood and artistic style of the film, and encourage audiences to see the movie. These dive…
A Temporal Variational Model for Story Generation
David Wilmot, Frank Keller
Recent language models can generate interesting and grammatically correct text in story generation but often lack plot development and long-term coherence. This paper experiments w…
A New Split for Evaluating True Zero-Shot Action Recognition
Shreyank N Gowda, Laura Sevilla-Lara, Kiyoon Kim +2
Zero-shot action recognition is the task of classifying action categories that are not available in the training set. In this setting, the standard evaluation protocol is to use ex…
CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition
Shreyank N Gowda, Laura Sevilla-Lara, Frank Keller +1
Zero-shot action recognition is the task of recognizingaction classes without visual examples, only with a seman-tic embedding which relates unseen to seen classes. Theproblem can…
Movie Summarization via Sparse Graph Construction
Pinelopi Papalampidi, Frank Keller, Mirella Lapata
We summarize full-length movies by creating shorter videos containing their most informative scenes. We explore the hypothesis that a summary can be created by assembling scenes wh…
Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads
Bowen Li, Taeuk Kim, Reinald Kim Amplayo +1
Transformer-based pre-trained language models (PLMs) have dramatically improved the state of the art in NLP across many tasks. This has led to substantial interest in analyzing the…
Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation
David Wilmot, Frank Keller
Suspense is a crucial ingredient of narrative fiction, engaging readers and making stories compelling. While there is a vast theoretical literature on suspense, it is computational…
Screenplay Summarization Using Latent Narrative Structure
Pinelopi Papalampidi, Frank Keller, Lea Frermann +1
Most general-purpose extractive summarization models are trained on news articles, which are short and present all important information upfront. As a result, such models are biase…
Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
Raffaella Bernardi, Ruket Cakici, Desmond Elliott +6
Automatic description generation from natural images is a challenging problem that has recently received a large amount of interest from the computer vision and natural language pr…