154 citations · 313 across the 22 of their papers we have counts for
4 papers · 1 filter
Do Bias Benchmarks Generalise? Evidence from Voice-based Evaluation of Gender Bias in SpeechLLMs
Shree Harsha Bokkahalli Satish, Gustav Eje Henter, Éva Székely
Recent work in benchmarking bias and fairness in speech large language models (SpeechLLMs) has relied heavily on multiple-choice question answering (MCQA) formats. The model is tas…
CARL-GT: Evaluating Causal Reasoning Capabilities of Large Language Models
Ruibo Tu, Hedvig Kjellström, Gustav Eje Henter +1
Causal reasoning capabilities are essential for large language models (LLMs) in a wide range of applications, such as education and healthcare. But there is still a lack of benchma…
Exploring Internal Numeracy in Language Models: A Case Study on ALBERT
Ulme Wennberg, Gustav Eje Henter
It has been found that Transformer-based language models have the ability to perform basic quantitative reasoning. In this paper, we propose a method for studying how these models…
The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models
Ulme Wennberg, Gustav Eje Henter
Mechanisms for encoding positional information are central for transformer-based language models. In this paper, we analyze the position embeddings of existing language models, fin…