8 citations · 25 across the 10 of their papers we have counts for
10 papers
Revisiting subword tokenization: A case study on affixal negation in large language models
Thinh Hung Truong, Yulia Otmakhova, Karin Verspoor +2
In this work, we measure the impact of affixal negation on modern English large language models (LLMs). In affixal negation, the negated meaning is expressed through a negative mor…
Multi-EuP: The Multilingual European Parliament Dataset for Analysis of Bias in Information Retrieval
Jinrui Yang, Timothy Baldwin, Trevor Cohn
We present Multi-EuP, a new multilingual benchmark dataset, comprising 22K multi-lingual documents collected from the European Parliament, spanning 24 languages. This dataset is de…
Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs
Yuxia Wang, Haonan Li, Xudong Han +2
With the rapid evolution of large language models (LLMs), new and hard-to-predict harmful capabilities are emerging. This requires developers to be able to identify risks through t…
Language models are not naysayers: An analysis of language models on negation benchmarks
Thinh Hung Truong, Timothy Baldwin, Karin Verspoor +1
Negation has been shown to be a major bottleneck for masked language models, such as BERT. However, whether this finding still holds for larger-sized auto-regressive language model…
Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP
Xudong Han, Timothy Baldwin, Trevor Cohn
Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. However current progress is hampered by a plurality of definitions…
Systematic Evaluation of Predictive Fairness
Xudong Han, Aili Shen, Trevor Cohn +2
Mitigating bias in training on biased datasets is an important open problem. Several techniques have been proposed, however the typical evaluation regime is very limited, consideri…
Optimising Equal Opportunity Fairness in Model Training
Aili Shen, Xudong Han, Trevor Cohn +2
Real-world datasets often encode stereotypes and societal biases. Such biases can be implicitly captured by trained models, leading to biased predictions and exacerbating existing…
fairlib: A Unified Framework for Assessing and Improving Classification Fairness
Xudong Han, Aili Shen, Yitong Li +3
This paper presents fairlib, an open-source framework for assessing and improving classification fairness. It provides a systematic framework for quickly reproducing existing basel…
Towards Equal Opportunity Fairness through Adversarial Learning
Xudong Han, Timothy Baldwin, Trevor Cohn
Adversarial training is a common approach for bias mitigation in natural language processing. Although most work on debiasing is motivated by equal opportunity, it is not explicitl…
Balancing out Bias: Achieving Fairness Through Balanced Training
Xudong Han, Timothy Baldwin, Trevor Cohn
Group bias in natural language processing tasks manifests as disparities in system error rates across texts authorized by different demographic groups, typically disadvantaging min…