8 papers
A Japanese Benchmark for Evaluating Social Bias in Reasoning Based on Attribution Theory
Taihei Shiotani, Masahiro Kaneko, Naoaki Okazaki
In enhancing the fairness of Large Language Models (LLMs), evaluating social biases rooted in the cultural contexts of specific linguistic regions is essential. However, most exist…
JUBAKU: An Adversarial Benchmark for Exposing Culturally Grounded Stereotypes in Japanese LLMs
Taihei Shiotani, Masahiro Kaneko, Ayana Niwa +4
Social biases reflected in language are inherently shaped by cultural norms, which vary significantly across regions and lead to diverse manifestations of stereotypes. Existing eva…
Machine Text Detectors are Membership Inference Attacks
Ryuto Koike, Liam Dugan, Masahiro Kaneko +2
Although membership inference attacks (MIAs) and machine-generated text detection target different goals, their methods often exploit similar signals based on a language model's pr…
GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human
Yuxia Wang, Artem Shelmanov, Jonibek Mansurov +23
We present the GenAI Content Detection Task~1 -- a shared task on binary machine generated text detection, conducted as a part of the GenAI workshop at COLING 2025. The task consis…
Balanced Multi-Factor In-Context Learning for Multilingual Large Language Models
Masahiro Kaneko, Alham Fikri Aji, Timothy Baldwin
Multilingual large language models (MLLMs) are able to leverage in-context learning (ICL) to achieve high performance by leveraging cross-lingual knowledge transfer without paramet…
Rectifying Belief Space via Unlearning to Harness LLMs' Reasoning
Ayana Niwa, Masahiro Kaneko, Kentaro Inui
Large language models (LLMs) can exhibit advanced reasoning yet still generate incorrect answers. We hypothesize that such errors frequently stem from spurious beliefs, proposition…