18 citations · 33 across the 10 of their papers we have counts for
10 papers
Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactions
Pedro Henrique Luz de Araujo, Michael A. Hedderich, Ali Modarressi +2
Persona-assigned large language models (LLMs) are used in domains such as education, healthcare, and sociodemographic simulation. Yet, they are typically evaluated only in short, s…
Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
Pedro Henrique Luz de Araujo, Paul Röttger, Dirk Hovy +1
Expert persona prompting -- assigning roles such as expert in math to language models -- is widely used for task improvement. However, prior work shows mixed results on its effecti…
Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles
Yuxi Xia, Pedro Henrique Luz de Araujo, Klim Zaporojets +1
Calibration, the alignment between model confidence and prediction accuracy, is critical for the reliable deployment of large language models (LLMs). Existing works neglect to meas…
Helpful assistant or fruitful facilitator? Investigating how personas affect language model behavior
Pedro Henrique Luz de Araujo, Benjamin Roth
One way to personalize and steer generations from large language models (LLM) is to assign a persona: a role that describes how the user expects the LLM to behave (e.g., a helpful…
Text-Guided Alternative Image Clustering
Andreas Stephan, Lukas Miklautz, Collin Leiber +4
Traditional image clustering techniques only find a single grouping within visual data. In particular, they do not provide a possibility to explicitly define multiple types of clus…
Exploring prompts to elicit memorization in masked language model-based named entity recognition
Yuxi Xia, Anastasiia Sedova, Pedro Henrique Luz de Araujo +3
Training data memorization in language models impacts model capability (generalization) and safety (privacy risk). This paper focuses on analyzing prompts' impact on detecting the…
Specification Overfitting in Artificial Intelligence
Benjamin Roth, Pedro Henrique Luz de Araujo, Yuxi Xia +2
Machine learning (ML) and artificial intelligence (AI) approaches are often criticized for their inherent bias and for their lack of control, accountability, and transparency. Cons…
Functionality learning through specification instructions
Pedro Henrique Luz de Araujo, Benjamin Roth
Test suites assess natural language processing models' performance on specific functionalities: cases of interest involving model robustness, fairness, or particular linguistic cap…
Cross-functional Analysis of Generalisation in Behavioural Learning
Pedro Henrique Luz de Araujo, Benjamin Roth
In behavioural testing, system functionalities underrepresented in the standard evaluation setting (with a held-out test set) are validated through controlled input-output pairs. O…
Checking HateCheck: a cross-functional analysis of behaviour-aware learning for hate speech detection
Pedro Henrique Luz de Araujo, Benjamin Roth
Behavioural testing -- verifying system capabilities by validating human-designed input-output pairs -- is an alternative evaluation method of natural language processing systems p…