69 citations · 127 across the 14 of their papers we have counts for
14 papers
Subjective ? On the Danger of Conflating Hate and Offence in Abusive Language Detection
Amanda Cercas Curry, Gavin Abercrombie, Zeerak Talat
Natural language processing research has begun to embrace the notion of annotator subjectivity, motivated by variations in labelling. This approach understands each annotator's vie…
Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution
Flor Miriam Plaza-del-Arco, Amanda Cercas Curry, Alba Curry +2
Large language models (LLMs) reflect societal norms and biases, especially about gender. While societal biases and stereotypes have been extensively researched in various NLP appli…
Understanding Counterspeech for Online Harm Mitigation
Yi-Ling Chung, Gavin Abercrombie, Florence Enock +2
Counterspeech offers direct rebuttals to hateful speech by challenging perpetrators of hate and showing support to targets of abuse. It provides a promising alternative to more con…
On the Origins of Bias in NLP through the Lens of the Jim Code
Fatma Elsafoury, Gavin Abercrombie
In this paper, we trace the biases in current natural language processing (NLP) models back to their origins in racism, sexism, and homophobia over the last 500 years. We review li…
iLab at SemEval-2023 Task 11 Le-Wi-Di: Modelling Disagreement or Modelling Perspectives?
Nikolas Vitsakis, Amit Parekh, Tanvi Dinkar +3
There are two competing approaches for modelling annotator disagreement: distributional soft-labelling approaches (which aim to capture the level of disagreement) or modelling pers…
Mirages: On Anthropomorphism in Dialogue Systems
Gavin Abercrombie, Amanda Cercas Curry, Tanvi Dinkar +2
Automated dialogue or conversational systems are anthropomorphised by developers and personified by users. While a degree of anthropomorphism may be inevitable due to the choice of…
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
Anya Belz, Craig Thomson, Ehud Reiter +39
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/le…
SemEval-2023 Task 11: Learning With Disagreements (LeWiDi)
Elisa Leonardelli, Alexandra Uma, Gavin Abercrombie +6
NLP datasets annotated with human judgments are rife with disagreements between the judges. This is especially true for tasks depending on subjective judgments such as sentiment an…
Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement
Gavin Abercrombie, Tanvi Dinkar, Amanda Cercas Curry +2
We commonly use agreement measures to assess the utility of judgements made by human annotators in Natural Language Processing (NLP) tasks. While inter-annotator agreement is frequ…
Risk-graded Safety for Handling Medical Queries in Conversational AI
Gavin Abercrombie, Verena Rieser
Conversational AI systems can engage in unsafe behaviour when handling users' medical queries that can have severe consequences and could even lead to deaths. Systems therefore nee…
ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AI
Amanda Cercas Curry, Gavin Abercrombie, Verena Rieser
We present the first English corpus study on abusive language towards three conversational AI systems gathered "in the wild": an open-domain social bot, a rule-based chatbot, and a…
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Emily Dinan, Gavin Abercrombie, A. Stevie Bergman +4
Over the last several years, end-to-end neural conversational agents have vastly improved in their ability to carry a chit-chat conversation with humans. However, these models are…
Alexa, Google, Siri: What are Your Pronouns? Gender and Anthropomorphism in the Design and Perception of Conversational Assistants
Gavin Abercrombie, Amanda Cercas Curry, Mugdha Pandya +1
Technology companies have produced varied responses to concerns about the effects of the design of their conversational AI systems. Some have claimed that their voice assistants ar…
Sentiment and position-taking analysis of parliamentary debates: A systematic literature review
Gavin Abercrombie, Riza Batista-Navarro
Parliamentary and legislative debate transcripts provide access to information concerning the opinions, positions and policy preferences of elected politicians. They attract attent…