185 citations · 582 across the 36 of their papers we have counts for
47 papers
NLP Verification: Towards a General Methodology for Certifying Robustness
Marco Casadio, Tanvi Dinkar, Ekaterina Komendantskaya +6
Machine Learning (ML) has exhibited substantial success in the field of Natural Language Processing (NLP). For example large language models have empirically proven to be capable o…
Multitask Multimodal Prompted Training for Interactive Embodied Task Completion
Georgios Pantazopoulos, Malvina Nikandrou, Amit Parekh +6
Interactive and embodied tasks pose at least two fundamental challenges to existing Vision & Language (VL) models, including 1) grounding language in trajectories of actions and ob…
FurNav: Development and Preliminary Study of a Robot Direction Giver
Bruce W. Wilson, Yann Schlosser, Rayane Tarkany +4
When giving directions to a lost-looking tourist, would you first reference the street-names, cardinal directions, landmarks, or simply tell them to walk five hundred metres in one…
Understanding Counterspeech for Online Harm Mitigation
Yi-Ling Chung, Gavin Abercrombie, Florence Enock +2
Counterspeech offers direct rebuttals to hateful speech by challenging perpetrators of hate and showing support to targets of abuse. It provides a promising alternative to more con…
The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering
Sabrina Chiesurin, Dimitris Dimakopoulos, Marco Antonio Sobrevilla Cabezudo +4
Large language models are known to produce output which sounds fluent and convincing, but is also often wrong, e.g. "unfaithful" with respect to a rationale as retrieved from a kno…
iLab at SemEval-2023 Task 11 Le-Wi-Di: Modelling Disagreement or Modelling Perspectives?
Nikolas Vitsakis, Amit Parekh, Tanvi Dinkar +3
There are two competing approaches for modelling annotator disagreement: distributional soft-labelling approaches (which aim to capture the level of disagreement) or modelling pers…
ANTONIO: Towards a Systematic Method of Generating NLP Benchmarks for Verification
Marco Casadio, Luca Arnaboldi, Matthew L. Daggitt +5
Verification of machine learning models used in Natural Language Processing (NLP) is known to be a hard problem. In particular, many known neural network verification methods that…
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
Anya Belz, Craig Thomson, Ehud Reiter +39
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/le…
Quality-agnostic Image Captioning to Safely Assist People with Vision Impairment
Lu Yu, Malvina Nikandrou, Jiali Jin +1
Automated image captioning has the potential to be a useful tool for people with vision impairments. Images taken by this user group are often noisy, which leads to incorrect and e…
SemEval-2023 Task 11: Learning With Disagreements (LeWiDi)
Elisa Leonardelli, Alexandra Uma, Gavin Abercrombie +6
NLP datasets annotated with human judgments are rife with disagreements between the judges. This is especially true for tasks depending on subjective judgments such as sentiment an…
Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement
Gavin Abercrombie, Tanvi Dinkar, Amanda Cercas Curry +2
We commonly use agreement measures to assess the utility of judgements made by human annotators in Natural Language Processing (NLP) tasks. While inter-annotator agreement is frequ…
Going for GOAL: A Resource for Grounded Football Commentaries
Alessandro Suglia, José Lopes, Emanuele Bastianelli +6
Recent video+language datasets cover domains where the interaction is highly structured, such as instructional videos, or where the interaction is scripted, such as TV shows. Both…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
Risk-graded Safety for Handling Medical Queries in Conversational AI
Gavin Abercrombie, Verena Rieser
Conversational AI systems can engage in unsafe behaviour when handling users' medical queries that can have severe consequences and could even lead to deaths. Systems therefore nee…
Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answering
Mavina Nikandrou, Lu Yu, Alessandro Suglia +2
Continual learning aims to train a model incrementally on a sequence of tasks without forgetting previous knowledge. Although continual learning has been widely studied in computer…
Why Robust Natural Language Understanding is a Challenge
Marco Casadio, Ekaterina Komendantskaya, Verena Rieser +4
With the proliferation of Deep Machine Learning into real-life applications, a particular property of this technology has been brought to attention: robustness Neural Networks noto…
Adversarial Robustness of Visual Dialog
Lu Yu, Verena Rieser
Adversarial robustness evaluates the worst-case performance scenario of a machine learning model to ensure its safety and reliability. This study is the first to investigate the ro…
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges
Shikib Mehri, Jinho Choi, Luis Fernando D'Haro +13
This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog. The workshop explored the current state of the art along with its limitations and suggeste…
MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization
Xinnuo Xu, Ondřej Dušek, Shashi Narayan +2
One of the most challenging aspects of current single-document news summarization is that the summary often contains 'extrinsic hallucinations', i.e., facts that are not present in…
ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AI
Amanda Cercas Curry, Gavin Abercrombie, Verena Rieser
We present the first English corpus study on abusive language towards three conversational AI systems gathered "in the wild": an open-domain social bot, a rule-based chatbot, and a…
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Emily Dinan, Gavin Abercrombie, A. Stevie Bergman +4
Over the last several years, end-to-end neural conversational agents have vastly improved in their ability to carry a chit-chat conversation with humans. However, these models are…
AGGGEN: Ordering and Aggregating while Generating
Xinnuo Xu, Ondřej Dušek, Verena Rieser +1
We present AGGGEN (pronounced 'again'), a data-to-text model which re-introduces two explicit sentence planning stages into neural data-to-text systems: input ordering and input ag…
Alexa, Google, Siri: What are Your Pronouns? Gender and Anthropomorphism in the Design and Perception of Conversational Assistants
Gavin Abercrombie, Amanda Cercas Curry, Mugdha Pandya +1
Technology companies have produced varied responses to concerns about the effects of the design of their conversational AI systems. Some have claimed that their voice assistants ar…
OTTers: One-turn Topic Transitions for Open-Domain Dialogue
Karin Sevegnani, David M. Howcroft, Ioannis Konstas +1
Mixed initiative in open-domain dialogue requires a system to pro-actively introduce new topics. The one-turn topic transition task explores how a system connects two topics in a c…