54 citations · 58 across the 12 of their papers we have counts for
12 papers
RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
Adithiyan Rajan Indira Saravanan, Kathleen C. Fraser
Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However, recent work has demonstrat…
From Perceived Effectiveness to Measured Impact: Identity-Aware Evaluation of Automated Counter-Stereotypes
Svetlana Kiritchenko, Anna Kerkhof, Isar Nejadgholi +1
We investigate the effect of automatically generated counter-stereotypes on gender bias held by users of various demographics on social media. Building on recent NLP advancements a…
Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency
Kathleen C. Fraser, Hillary Dawkins, Isar Nejadgholi +1
Fine-tuning a general-purpose large language model (LLM) for a specific domain or task has become a routine procedure for ordinary users. However, fine-tuning is known to remove th…
When Detection Fails: The Power of Fine-Tuned Models to Generate Human-Like Social Media Text
Hillary Dawkins, Kathleen C. Fraser, Svetlana Kiritchenko
Detecting AI-generated text is a difficult problem to begin with; detecting AI-generated text on social media is made even more difficult due to the short text length and informal,…
Tackling Social Bias against the Poor: A Dataset and Taxonomy on Aporophobia
Georgina Curto, Svetlana Kiritchenko, Muhammad Hammad Fahim Siddiqui +2
Eradicating poverty is the first goal in the United Nations Sustainable Development Goals. However, aporophobia -- the societal bias against people living in poverty -- constitutes…
Adaptable Moral Stances of Large Language Models on Sexist Content: Implications for Society and Gender Discourse
Rongchen Guo, Isar Nejadgholi, Hillary Dawkins +2
This work provides an explanatory view of how LLMs can apply moral reasoning to both criticize and defend sexist language. We assessed eight large language models, all of which dem…
Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods
Kathleen C. Fraser, Hillary Dawkins, Svetlana Kiritchenko
Large language models (LLMs) have advanced to a point that even humans have difficulty discerning whether a text was generated by another human, or by a computer. However, knowing…
Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals
Phillip Howard, Kathleen C. Fraser, Anahita Bhiwandiwalla +1
With the advent of Large Language Models (LLMs) possessing increasingly impressive capabilities, a number of Large Vision-Language Models (LVLMs) have been proposed to augment LLMs…
Challenging Negative Gender Stereotypes: A Study on the Effectiveness of Automated Counter-Stereotypes
Isar Nejadgholi, Kathleen C. Fraser, Anna Kerkhof +1
Gender stereotypes are pervasive beliefs about individuals based on their gender that play a significant role in shaping societal attitudes, behaviours, and even opportunities. Rec…
Uncovering Bias in Large Vision-Language Models with Counterfactuals
Phillip Howard, Anahita Bhiwandiwalla, Kathleen C. Fraser +1
With the advent of Large Language Models (LLMs) possessing increasingly impressive capabilities, a number of Large Vision-Language Models (LVLMs) have been proposed to augment LLMs…
Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Images
Kathleen C. Fraser, Svetlana Kiritchenko
Following on recent advances in large language models (LLMs) and subsequent chat models, a new wave of large vision-language models (LVLMs) has emerged. Such models can incorporate…
The crime of being poor
Georgina Curto, Svetlana Kiritchenko, Isar Nejadgholi +1
The criminalization of poverty has been widely denounced as a collective bias against the most vulnerable. NGOs and international organizations claim that the poor are blamed for t…