6 papers
Evaluation Revisited: A Taxonomy of Evaluation Concerns in Natural Language Processing
Ruchira Dhar, Anders Søgaard
Recent advances in large language models (LLMs) have prompted a growing body of work that questions the methodology of prevailing evaluation practices. However, many such critiques…
Evaluating Adjective-Noun Compositionality in LLMs: Functional vs Representational Perspectives
Ruchira Dhar, Qiwei Peng, Anders Søgaard
Compositionality is considered central to language abilities. As performant language systems, how do large language models (LLMs) do on compositional tasks? We evaluate adjective-n…
Understanding Subword Compositionality of Large Language Models
Qiwei Peng, Yekun Chai, Anders Søgaard
Large language models (LLMs) take sequences of subwords as input, requiring them to effective compose subword representations into meaningful word-level representations. In this pa…
Debiasing Multilingual LLMs in Cross-lingual Latent Space
Qiwei Peng, Guimin Hu, Yekun Chai +1
Debiasing techniques such as SentDebias aim to reduce bias in large language models (LLMs). Previous studies have evaluated their cross-lingual transferability by directly applying…
SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval
Qiwei Peng, Robert Moro, Michal Gregor +7
The rapid spread of online disinformation presents a global challenge, and machine learning has been widely explored as a potential solution. However, multilingual settings and low…
Revisiting the Othello World Model Hypothesis
Yifei Yuan, Anders Søgaard
Li et al. (2023) used the Othello board game as a test case for the ability of GPT-2 to induce world models, and were followed up by Nanda et al. (2023b). We briefly discuss the or…