1 citations · 1 across the 8 of their papers we have counts for
9 papers
Explicit Boundary Markers for Subword Vocabularies
Sander Land, Clara Meister
Subword tokenizers represent many common words twice in space-using writing systems, once with a leading space and once without. The two entries have separate embeddings in models,…
MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment
Sander Land
The Unigram tokenizer uses an elegant representation which makes it straightforward to edit vocabularies, but its training is comparatively heavy and complex. We introduce MinGram…
Auditing LLM Benchmarks with Item Response Theory
Sander Land, Daniel M. Bikel
LLM benchmark labels are frozen at release and silently propagated into downstream benchmarks, errors and all. We introduce an Item Response Theory-based indicator that surfaces li…
Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens
Zhenyu Zhao, Sander Land, Daniel M. Bikel +1
Reasoning in Large Language Models incurs significant inference-time compute, yet the token-level information structure of reasoning traces remains underexplored. We observe that r…
Which Pieces Does Unigram Tokenization Really Need?
Sander Land, Yuval Pinter
The Unigram tokenization algorithm offers a probabilistic alternative to the greedy heuristics of Byte-Pair Encoding. Despite its theoretical elegance, its implementation in practi…
RewardBench 2: Advancing Reward Model Evaluation
Saumya Malik, Valentina Pyatkin, Sander Land +4
Reward models are used throughout the post-training of language models to capture nuanced signals from preference data and provide a training target for optimization across instruc…
Command A: An Enterprise-Ready Large Language Model
Team Cohere, :, Aakanksha +227
In this report we describe the development of Command A, a powerful large language model purpose-built to excel at real-world enterprise use cases. Command A is an agent-optimised…
Understanding Likelihood Over-optimisation in Direct Alignment Algorithms
Zhengyan Shi, Sander Land, Acyr Locatelli +2
Direct Alignment Algorithms (DAAs), such as Direct Preference Optimisation (DPO) and Identity Preference Optimisation (IPO), have emerged as alternatives to online Reinforcement Le…
Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models
Sander Land, Max Bartolo
The disconnect between tokenizer creation and model training in language models allows for specific inputs, such as the infamous SolidGoldMagikarp token, to induce unwanted model b…