7 papers
FLEXITOKENS: Flexible Tokenization for Evolving Language Models
Abraham Toluwase Owodunni, Orevaoghene Ahia, Sachin Kumar
Adapting language models to new data distributions by simple finetuning is challenging. This is due to the rigidity of their subword tokenizers, which typically remain unchanged du…
IDIOLEX: Unified and Continuous Representations for Idiolectal and Stylistic Variation
Anjali Kantharuban, Aarohi Srivastava, Fahim Faisal +5
Existing sentence representations primarily encode what a sentence says, rather than how it is expressed, even though the latter is important for many applications. In contrast, we…
Encode Once, Decode Never: Reusing Audio LM Internals for Efficient Temporal Localization
Joesph An, Joseph An, Phillip Keung +3
Audio language models process input audio into rich frame-level representations, but the standard approach to temporal localization generates timestamps as sequences of text tokens…
BASS: Benchmarking Audio LMs for Musical Structure and Semantic Reasoning
Min Jang, Orevaoghene Ahia, Nazif Tamer +3
Music understanding is a complex task that often requires reasoning over both structural and semantic elements of audio. We introduce BASS, designed to evaluate music understanding…
Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
Brian Siyuan Zheng, Alisa Liu, Orevaoghene Ahia +3
Modern tokenizers employ deterministic algorithms to map text into a single "canonical" token sequence, yet the same string can be encoded as many non-canonical tokenizations using…
Cognitive Foundations for Reasoning and Their Manifestation in LLMs
Priyanka Kargupta, Shuyue Stella Li, Haocheng Wang +9
Large language models (LLMs) solve complex problems yet fail on simpler variants, suggesting they achieve correct outputs through mechanisms fundamentally different from human reas…