77 citations · 101 across the 18 of their papers we have counts for
22 papers
What, When, and How: Audio Description as Constrained Global Optimization
Igor Sterner, Mirella Lapata, Alex Lascarides +1
Audio Description (AD) makes movies accessible to blind and visually impaired audiences by narrating visual information in gaps between dialogue. Existing automatic AD systems larg…
Learning to Assemble Novel Structures with Unfamiliar Parts under Semantic Constraints
Jonghyuk Park, Alex Lascarides, Subramanian Ramamoorthy
This paper describes a neurosymbolic architecture for learning to assemble novel structures using evidence from embodied conversations and task demonstrations. We focus on scenario…
Comparing British and American Audio Description of Movies
Igor Sterner, Alex Lascarides, Frank Keller
Narrating the visual component of movies is known as audio description. It is a narrative technique designed to enable blind and visually impaired individuals to follow the story.…
REFRAMED: Towards Realistic Audio Description Generation for Movies
Igor Sterner, Mirella Lapata, Alex Lascarides +1
Audio Description (AD) is a verbal narration of key visual content in videos, enabling access for visually impaired audiences. Unlike standard video captioning, AD is a structured…
Contrastive Learning with Narrative Twins for Modeling Story Salience
Igor Sterner, Alex Lascarides, Frank Keller
Understanding narratives requires identifying which events are most salient for a story's progression. We present a contrastive learning framework for modeling narrative salience t…
: How to learn from procedural How-to questions
Gautier Dagan, Frank Keller, Alex Lascarides
An agent facing a planning problem can use answers to how-to questions to reduce uncertainty and fill knowledge gaps, helping it solve both current and future tasks. However, their…
Learning Visually Grounded Domain Ontologies via Embodied Conversation and Explanation
Jonghyuk Park, Alex Lascarides, Subramanian Ramamoorthy
In this paper, we offer a learning framework in which the agent's knowledge gaps are overcome through corrective feedback from a teacher whenever the agent explains its (incorrect)…
Plancraft: an evaluation dataset for planning with LLM agents
Gautier Dagan, Frank Keller, Alex Lascarides
We present Plancraft, a multi-modal evaluation dataset for LLM agents. Plancraft has both a text-only and multi-modal interface, based on the Minecraft crafting GUI. We include the…
Conversational Code Generation: a Case Study of Designing a Dialogue System for Generating Driving Scenarios for Testing Autonomous Vehicles
Rimvydas Rubavicius, Antonio Valerio Miceli-Barone, Alex Lascarides +1
Cyber-physical systems like autonomous vehicles are tested in simulation before deployment, using domain-specific programs for scenario specification. To aid the testing of autonom…
SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning
Rimvydas Rubavicius, Peter David Fagan, Alex Lascarides +1
This paper addresses a challenging interactive task learning scenario we call rearrangement under unawareness: an agent must manipulate a rigid-body environment without knowing a k…
Dialogue-based generation of self-driving simulation scenarios using Large Language Models
Antonio Valerio Miceli-Barone, Alex Lascarides, Craig Innes
Simulation is an invaluable tool for developing and evaluating controllers for self-driving cars. Current simulation frameworks are driven by highly-specialist domain specific lang…
Dynamic Planning with a LLM
Gautier Dagan, Frank Keller, Alex Lascarides
While Large Language Models (LLMs) can solve many NLP tasks in zero-shot settings, applications involving embodied agents remain problematic. In particular, complex plans that requ…
Interactive Acquisition of Fine-grained Visual Concepts by Exploiting Semantics of Generic Characterizations in Discourse
Jonghyuk Park, Alex Lascarides, Subramanian Ramamoorthy
Interactive Task Learning (ITL) concerns learning about unforeseen domain concepts via natural interactions with human users. The learner faces a number of significant constraints:…
Learning Manner of Execution from Partial Corrections
Mattias Appelgren, Alex Lascarides
Some actions must be executed in different ways depending on the context. For example, wiping away marker requires vigorous force while wiping away almonds requires more gentle for…
Learning the Effects of Physical Actions in a Multi-modal Environment
Gautier Dagan, Frank Keller, Alex Lascarides
Large Language Models (LLMs) handle physical commonsense information inadequately. As a result of being trained in a disembodied setting, LLMs often fail to predict an action's out…
Disentangled Relational Representations for Explaining and Learning from Demonstration
Yordan Hristov, Daniel Angelov, Michael Burke +2
Learning from demonstration is an effective method for human users to instruct desired robot behaviour. However, for most non-trivial tasks of practical interest, efficient learnin…
Learning Factored Markov Decision Processes with Unawareness
Craig Innes, Alex Lascarides
Methods for learning and planning in sequential decision problems often assume the learner is aware of all possible states and actions in advance. This assumption is sometimes unte…
Interpretable Latent Spaces for Learning from Demonstration
Yordan Hristov, Alex Lascarides, Subramanian Ramamoorthy
Effective human-robot interaction, such as in robot learning from human demonstration, requires the learning agent to be able to ground abstract concepts (such as those contained w…
Reasoning about Unforeseen Possibilities During Policy Learning
Craig Innes, Alex Lascarides, Stefano V Albrecht +2
Methods for learning optimal policies in autonomous agents often assume that the way the domain is conceptualised---its possible states and actions and their causal structure---is…
Grounding Symbols in Multi-Modal Instructions
Yordan Hristov, Svetlin Penkov, Alex Lascarides +1
As robots begin to cohabit with humans in semi-structured environments, the need arises to understand instructions involving rich variability---for instance, learning to ground sym…
Learning Sentence-internal Temporal Relations
M. Lapata, A. Lascarides
In this paper we propose a data intensive approach for inferring sentence-internal temporal relations. Temporal inference is relevant for practical NLP applications which either ex…
Intentions and Information in Discourse
Nicholas Asher, Alex Lascarides
This paper is about the flow of inference between communicative intentions, discourse structure and the domain during discourse processing. We augment a theory of discourse interpr…