1 citations · 3 across the 10 of their papers we have counts for
8 papers · 1 filter
IDIOLEX: Unified and Continuous Representations for Idiolectal and Stylistic Variation
Anjali Kantharuban, Aarohi Srivastava, Fahim Faisal +5
Existing sentence representations primarily encode what a sentence says, rather than how it is expressed, even though the latter is important for many applications. In contrast, we…
FLEXITOKENS: Flexible Tokenization for Evolving Language Models
Abraham Toluwase Owodunni, Orevaoghene Ahia, Sachin Kumar
Adapting language models to new data distributions by simple finetuning is challenging. This is due to the rigidity of their subword tokenizers, which typically remain unchanged du…
Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
Brian Siyuan Zheng, Alisa Liu, Orevaoghene Ahia +3
Modern tokenizers employ deterministic algorithms to map text into a single "canonical" token sequence, yet the same string can be encoded as many non-canonical tokenizations using…
MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization
Orevaoghene Ahia, Sachin Kumar, Hila Gonen +4
In multilingual settings, non-Latin scripts and low-resource languages are usually disadvantaged in terms of language models' utility, efficiency, and cost. Specifically, previous…
Voices Unheard: NLP Resources and Models for Yorùbá Regional Dialects
Orevaoghene Ahia, Anuoluwapo Aremu, Diana Abagyan +5
Yorùbá an African language with roughly 47 million speakers encompasses a continuum with several dialects. Recent efforts to develop NLP technologies for African languages have foc…
Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning
Everlyn Asiko Chimoto, Jay Gala, Orevaoghene Ahia +3
Neural Machine Translation models are extremely data and compute-hungry. However, not all data points contribute equally to model training and generalization. Data pruning to remov…