23 citations · 69 across the 19 of their papers we have counts for
20 papers
Benchmarking Gender and Political Bias in Large Language Models
Jinrui Yang, Xudong Han, Timothy Baldwin
We introduce EuroParlVote, a novel benchmark for evaluating large language models (LLMs) in politically sensitive contexts. It links European Parliament debate speeches to roll-cal…
ParlAI Vote: A Web Platform for Analyzing Gender and Political Bias in Large Language Models
Wenjie Lin, Hange Liu, Yingying Zhuang +5
We present ParlAI Vote, an interactive web platform for exploring European Parliament debates and votes, and for testing LLMs on vote prediction and bias analysis. This web system…
RuozhiBench: Evaluating LLMs with Logical Fallacies and Misleading Premises
Zenan Zhai, Hao Li, Xudong Han +4
Recent advances in large language models (LLMs) have shown that they can answer questions requiring complex reasoning. However, their ability to identify and respond to text contai…
ToolGen: Unified Tool Retrieval and Calling via Generation
Renxi Wang, Xudong Han, Lei Ji +3
As large language models (LLMs) advance, their inability to autonomously execute tasks by directly interacting with external tools remains a critical limitation. Traditional method…
Against The Achilles' Heel: A Survey on Red Teaming for Generative Models
Lizhi Lin, Honglin Mu, Zenan Zhai +9
Generative models are rapidly gaining popularity and being integrated into everyday applications, raising concerns over their safe use as various vulnerabilities are exposed. In li…
A Chinese Dataset for Evaluating the Safeguards in Large Language Models
Yuxia Wang, Zenan Zhai, Haonan Li +6
Many studies have demonstrated that large language models (LLMs) can produce harmful responses, exposing users to unexpected risks when LLMs are deployed. Previous studies have pro…
Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents
Renxi Wang, Haonan Li, Xudong Han +2
Large language models (LLMs) have achieved success in acting as agents, which interact with environments through tools such as search engines. However, LLMs are optimized for langu…
Demystifying Instruction Mixing for Fine-tuning Large Language Models
Renxi Wang, Haonan Li, Minghao Wu +4
Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. However, the procedure to optimizing the mixing of instruction datas…
Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models
Neha Sengupta, Sunil Kumar Sahu, Bokang Jia +29
We introduce Jais and Jais-chat, new state-of-the-art Arabic-centric foundation and instruction-tuned open generative large language models (LLMs). The models are based on the GPT-…
Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs
Yuxia Wang, Haonan Li, Xudong Han +2
With the rapid evolution of large language models (LLMs), new and hard-to-predict harmful capabilities are emerging. This requires developers to be able to identify risks through t…
Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP
Xudong Han, Timothy Baldwin, Trevor Cohn
Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. However current progress is hampered by a plurality of definitions…
Systematic Evaluation of Predictive Fairness
Xudong Han, Aili Shen, Trevor Cohn +2
Mitigating bias in training on biased datasets is an important open problem. Several techniques have been proposed, however the typical evaluation regime is very limited, consideri…
Optimising Equal Opportunity Fairness in Model Training
Aili Shen, Xudong Han, Trevor Cohn +2
Real-world datasets often encode stereotypes and societal biases. Such biases can be implicitly captured by trained models, leading to biased predictions and exacerbating existing…
fairlib: A Unified Framework for Assessing and Improving Classification Fairness
Xudong Han, Aili Shen, Yitong Li +3
This paper presents fairlib, an open-source framework for assessing and improving classification fairness. It provides a systematic framework for quickly reproducing existing basel…
Towards Equal Opportunity Fairness through Adversarial Learning
Xudong Han, Timothy Baldwin, Trevor Cohn
Adversarial training is a common approach for bias mitigation in natural language processing. Although most work on debiasing is motivated by equal opportunity, it is not explicitl…
Contrastive Learning for Fair Representations
Aili Shen, Xudong Han, Trevor Cohn +2
Trained classification models can unintentionally lead to biased representations and predictions, which can reinforce societal preconceptions and stereotypes. Existing debiasing me…
Evaluating Debiasing Techniques for Intersectional Biases
Shivashankar Subramanian, Xudong Han, Timothy Baldwin +2
Bias is pervasive in NLP models, motivating the development of automatic debiasing techniques. Evaluation of NLP debiasing methods has largely been limited to binary attributes in…
Balancing out Bias: Achieving Fairness Through Balanced Training
Xudong Han, Timothy Baldwin, Trevor Cohn
Group bias in natural language processing tasks manifests as disparities in system error rates across texts authorized by different demographic groups, typically disadvantaging min…
Diverse Adversaries for Mitigating Bias in Training
Xudong Han, Timothy Baldwin, Trevor Cohn
Adversarial learning can learn fairer and less biased models of language than standard methods. However, current adversarial techniques only partially mitigate model bias, added to…
Grounding learning of modifier dynamics: An application to color naming
Xudong Han, Philip Schulz, Trevor Cohn
Grounding is crucial for natural language understanding. An important subtask is to understand modified color expressions, such as 'dirty blue'. We present a model of color modifie…