5 papers
FTB Graph: Determining and Validating First-token Broadcasters and Language-Identity Head Circuits in Multilingual Language Models
Arjun Pillai, Christian Hoang, Anjelo Laroza
Large language models operating in multilingual contexts must resolve target response languages early in generation, yet the causal circuitry governing first-token language identit…
Meddies-PII: A Multilingual Framework for Personally Identifiable Information Extraction in Clinical De-identification
Linh Uyen Le, Christian Hoang, Ho Dinh Tri +1
Clinical de-identification relies on accurately identifying personally identifiable information (PII). However, manually annotated datasets are costly to construct, while existing…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
How Perturbations Propagate: A Multi-Level Analysis of Robustness in Large Language Models
Dun Li Chan, Emily Liu, Niyathi Allu +1
Language models encounter typos, corrupted text, altered words, and disrupted token order, yet robustness is usually evaluated only through output behavior. We study how six natura…
First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers
Arjun Pillai, Christian Hoang, Anjelo Jann Laroza
Why do multilingual language models sometimes generate in the wrong language, and why is this so hard to fix? We introduce Language Identity Head Ablation (LIHA), a causal interven…