8 papers
No Single Tokenizer Feature Reliably Predicts Downstream Language Model Performance
Antonio Lopardo, Luis Frentzen Salim, Shikhar Srivastava +3
Tokenization is a critical component of the language modeling pipeline, but it is poorly understood. Tokenizers can impact downstream performance, but there is currently mixed evid…
SIGN-MAPPO: State-Intent-Aware Graph Multi-Agent Reinforcement Learning for Cooperative Driving under Imperfect V2X
Zide Fan, Jiahui Yuan, Kaixin Zhang +3
Cooperative driving at mixed-traffic unsignalized intersections is challenged by imperfect vehicle-to-everything (V2X) communication, human-driven vehicle (HV) intentions that cann…
Walk-on-Ambient-Patch: Solver-Adapted Isometric Embeddings for Projected Walk on Spheres
Yunfeng Liao, Xiucheng Li
Projected Walk on Spheres (PWoS) solves surface PDEs through ambient sampling and closest-point projection, but tightly folded embeddings can force short steps or ambiguous project…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It
Quang Minh Nguyen, Luis Frentzen Salim
Humans naturally form and express beliefs in daily communication, e.g., "I think the answer is 3" or "I suppose that's right." Such beliefs inevitably intertwine with fact and know…
Positional Cognitive Specialization: Where Do LLMs Learn To Comprehend and Speak Your Language?
Luis Frentzen Salim, Lun-Wei Ku, Hsing-Kuo Kenneth Pao
Adapting large language models (LLMs) to new languages is an expensive and opaque process. Understanding how language models acquire new languages and multilingual abilities is key…
Beyond Many-Shot Translation: Scaling In-Context Demonstrations For Low-Resource Machine Translation
Luis Frentzen Salim, Esteban Carlin, Alexandre Morinvil +2
Building machine translation (MT) systems for low-resource languages is notably difficult due to the scarcity of high-quality data. Although Large Language Models (LLMs) have impro…
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett +94
Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heter…