65 citations · 192 across the 33 of their papers we have counts for
40 papers
FedMental: Evaluating Federated Learning for Mental Health Detection from Social Media Data
Nuredin Ali Abdelkadir, Anjali Ratnam, Zeerak Talat +1
Social media text data are often used to train Machine Learning (ML) models to identify users exhibiting high-risk mental health behaviors. However, sharing this sensitive data pos…
Big AI's Regulatory Capture: Mapping Industry Interference and Government Complicity
Abeba Birhane, Riccardo Angius, William Agnew +4
Over the past decade, the AI industry has come to exert an unprecedented economic, political and societal power and influence. It is therefore critical that we comprehend the exten…
Aligning Stuttered-Speech Research with End-User Needs: Scoping Review, Survey, and Guidelines
Hawau Olamide Toyin, Mutiah Apampa, Toluwani Aremu +6
Atypical speech is receiving greater attention in speech technology research, but much of this work unfolds with limited interdisciplinary dialogue. For stuttered speech in particu…
Who Gets Heard? Rethinking Fairness in AI for Music Systems
Atharva Mehta, Shivam Chauhan, Megha Sharma +5
In recent years, the music research community has examined risks of AI models for music, with generative AI models in particular, raised concerns about copyright, deepfakes, and tr…
Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations
Anka Reuel, Avijit Ghosh, Jenny Chim +32
Foundation models are increasingly central to high-stakes AI systems, and governance frameworks now depend on evaluations to assess their risks and capabilities. Although general c…
Online Learning Defense against Iterative Jailbreak Attacks via Prompt Optimization
Masahiro Kaneko, Zeerak Talat, Timothy Baldwin
Iterative jailbreak methods that repeatedly rewrite and input prompts into large language models (LLMs) to induce harmful outputs -- using the model's previous responses to guide e…
Personal Attribute Leakage in Federated Speech Models
Hamdan Al-Ali, Ali Reza Ghavamipour, Tommaso Caselli +3
Federated learning is a common method for privacy-preserving training of machine learning models. In this paper, we analyze the vulnerability of ASR models to attribute inference a…
Code-Switching in End-to-End Automatic Speech Recognition: A Systematic Literature Review
Maha Tufail Agro, Atharva Kulkarni, Karima Kadaoui +2
Motivated by a growing research interest into automatic speech recognition (ASR), and the growing body of work for languages in which code-switching (CS) often occurs, we present a…
IYKYK: Using language models to decode extremist cryptolects
Christine de Kock, Arij Riabi, Zeerak Talat +3
Extremist groups develop complex in-group language, also referred to as cryptolects, to exclude or mislead outsiders. We investigate the ability of current language technologies to…
Exploitation All the Way Down: Calling out the Root Cause of Bad Online Experiences for Users of the "Majority World"
Hellina Hailu Nigatu, Zeerak Talat
Global Majority users are exposed to multitudes of harm when interacting with online platforms. This essay illuminates how exploitation in the advances of Artificial Intelligence i…
The Only Way is Ethics: A Guide to Ethical Research with Large Language Models
Eddie L. Ungless, Nikolas Vitsakis, Zeerak Talat +5
There is a significant body of work looking at the ethical considerations of large language models (LLMs): critiquing tools to measure performance and harms; proposing toolkits to…
A Capabilities Approach to Studying Bias and Harm in Language Technologies
Hellina Hailu Nigatu, Zeerak Talat
Mainstream Natural Language Processing (NLP) research has ignored the majority of the world's languages. In moving from excluding the majority of the world's languages to blindly a…
Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models
Eddie L. Ungless, Nikolas Vitsakis, Zeerak Talat +5
This whitepaper offers an overview of the ethical considerations surrounding research into or with large language models (LLMs). As LLMs become more integrated into widely used app…
Understanding "Democratization" in NLP and ML Research
Arjun Subramonian, Vagrant Gautam, Dietrich Klakow +1
Recent improvements in natural language processing (NLP) and machine learning (ML) and increased mainstream adoption have led to researchers frequently discussing the "democratizat…
Exploring the Limitations of Detecting Machine-Generated Text
Jad Doughman, Osama Mohammed Afzal, Hawau Olamide Toyin +3
Recent improvements in the quality of the generations by large language models have spurred research into identifying machine-generated text. Such work often presents high-performi…
The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels
Eve Fleisig, Su Lin Blodgett, Dan Klein +1
Longstanding data labeling practices in machine learning involve collecting and aggregating labels from multiple annotators. But what should we do when annotators disagree? Though…
Classist Tools: Social Class Correlates with Performance in NLP
Amanda Cercas Curry, Giuseppe Attanasio, Zeerak Talat +1
Since the foundational work of William Labov on the social stratification of language (Labov, 1964), linguistics has made concentrated efforts to explore the links between sociodem…
Impoverished Language Technology: The Lack of (Social) Class in NLP
Amanda Cercas Curry, Zeerak Talat, Dirk Hovy
Since Labov's (1964) foundational work on the social stratification of language, linguistics has dedicated concerted efforts towards understanding the relationships between socio-d…
Subjective ? On the Danger of Conflating Hate and Offence in Abusive Language Detection
Amanda Cercas Curry, Gavin Abercrombie, Zeerak Talat
Natural language processing research has begun to embrace the notion of annotator subjectivity, motivated by variations in labelling. This approach understands each annotator's vie…
Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon
Fajri Koto, Tilman Beck, Zeerak Talat +2
Improving multilingual language models capabilities in low-resource languages is generally difficult due to the scarcity of large-scale data in those languages. In this paper, we r…
Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms
Organizers of QueerInAI, Nathan Dennler, Anaelia Ovalle +11
Bias evaluation benchmarks and dataset and model documentation have emerged as central processes for assessing the biases and harms of artificial intelligence (AI) systems. However…
Evaluating the Social Impact of Generative AI Systems in Systems and Society
Irene Solaiman, Zeerak Talat, William Agnew +28
Generative AI systems across modalities, ranging from text (including code), image, audio, and video, have broad social impacts, but there is no official standard for means of eval…
Mirages: On Anthropomorphism in Dialogue Systems
Gavin Abercrombie, Amanda Cercas Curry, Tanvi Dinkar +2
Automated dialogue or conversational systems are anthropomorphised by developers and personified by users. While a degree of anthropomorphism may be inevitable due to the choice of…
Thorny Roses: Investigating the Dual Use Dilemma in Natural Language Processing
Lucie-Aimée Kaffee, Arnav Arora, Zeerak Talat +1
Dual use, the intentional, harmful reuse of technology and scientific artefacts, is a problem yet to be well-defined within the context of Natural Language Processing (NLP). Howeve…