4 papers · 1 filter
A Probabilistic Interpretation of KV Cache Eviction
Renato Geh, Alex Chen, Daniel Israel +2
The premise and promise of KV (cache) eviction is simple: higher throughput can be achieved by evicting some entries from the KV cache, at a negligible cost to quality. This holds…
Probabilistic Programs of Thought
Poorva Garg, Renato Lui Geh, Daniel Israel +3
LLMs are widely used for code generation and mathematical reasoning tasks where they are required to generate structured output. They either need to reason about code, generate cod…
Adversarial Tokenization
Renato Lui Geh, Zilei Shao, Guy Van den Broeck
Current LLM pipelines account for only one possible tokenization for a given string, ignoring exponentially many alternative tokenizations during training and inference. For exampl…
Where is the signal in tokenization space?
Renato Lui Geh, Honghua Zhang, Kareem Ahmed +2
Large Language Models (LLMs) are typically shipped with tokenizers that deterministically encode text into so-called canonical token sequences, to which the LLMs assign probability…