68 papers
DuDi: Dual-Signal Distillation with Cross-Lingual Verbalizer
Patomporn Payoungkhamdee, Tinnakit Udsa, Jian Gang Ngui +3
Small language models (SLMs) are efficient and scalable, but their multilingual capabilities degrade severely at sub-billion scales, especially for Southeast Asian (SEA) languages.…
TimpaTeks: Automatic In-place Text Sequence Modification via Diffusion Language Model Steering
Ryandito Diandaru, Ikhlasul Akmal Hanif, Fadli Aulawi Al Ghiffari +2
We extend activation steering to diffusion language models (DLMs) and study a novel problem that arose due to the inference mechanism of DLMs: Modifying a text in-place to manifest…
SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding
Peerawat Chomphooyod, Jian Gang Ngui, Yosephine Susanto +5
Frontier LLMs perform well in Western contexts, but remain poorly tested on underrepresented cultures such as those in Southeast Asia (SEA). Existing NLI benchmarks are largely Wes…
Lessons from the Trenches on Reproducible Evaluation of Language Models
Stella Biderman, Hailey Schoelkopf, Lintang Sutawika +27
Reliable evaluation of language models (LMs) remains an open challenge. Re- searchers and engineers face methodological issues such as the sensitivity of models to evaluation setup…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Sense Representations Are Inducible Interfaces
Jan Christian Blaise Cruz, Alham Fikri Aji
Sense representations (explicit, per-token meaning decompositions) are useful for disambiguation, steering, and cross-lingual alignment, but existing approaches require models to b…