5 papers · 1 filter
CROPE: Evaluating In-Context Adaptation of Vision and Language Models to Culture-Specific Concepts
Malvina Nikandrou, Georgios Pantazopoulos, Nikolas Vitsakis +2
As Vision and Language models (VLMs) are reaching users across the globe, assessing their cultural understanding has become a critical challenge. In this paper, we introduce CROPE,…
The Only Way is Ethics: A Guide to Ethical Research with Large Language Models
Eddie L. Ungless, Nikolas Vitsakis, Zeerak Talat +5
There is a significant body of work looking at the ethical considerations of large language models (LLMs): critiquing tools to measure performance and harms; proposing toolkits to…
Investigating the Role of Instruction Variety and Task Difficulty in Robotic Manipulation Tasks
Amit Parekh, Nikolas Vitsakis, Alessandro Suglia +1
Evaluating the generalisation capabilities of multimodal models based solely on their performance on out-of-distribution data fails to capture their true robustness. This work intr…
Re-examining Sexism and Misogyny Classification with Annotator Attitudes
Aiqi Jiang, Nikolas Vitsakis, Tanvi Dinkar +2
Gender-Based Violence (GBV) is an increasing problem online, but existing datasets fail to capture the plurality of possible annotator perspectives or ensure the representation of…
Voices in a Crowd: Searching for Clusters of Unique Perspectives
Nikolas Vitsakis, Amit Parekh, Ioannis Konstas
Language models have been shown to reproduce underlying biases existing in their training data, which is the majority perspective by default. Proposed solutions aim to capture mino…