4 papers
Meddies-PII: A Multilingual Framework for Personally Identifiable Information Extraction in Clinical De-identification
Linh Uyen Le, Christian Hoang, Huy Hoang Ha
Clinical de-identification relies on accurately identifying personally identifiable information (PII). However, manually annotated datasets are costly to construct, while existing…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +241
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
How Perturbations Propagate: A Multi-Level Analysis of Robustness in Large Language Models
Dun Li Chan, Emily Liu, Niyathi Allu +1
Language models encounter typos, corrupted text, altered words, and disrupted token order, yet robustness is usually evaluated only through output behavior. We study how six natura…
First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers
Arjun Pillai, Christian Hoang, Anjelo Jann Laroza
Why do multilingual language models sometimes generate in the wrong language, and why is this so hard to fix? We introduce Language Identity Head Ablation (LIHA), a causal interven…