6 papers
A Probabilistic Interpretation of KV Cache Eviction
Renato Geh, Alex Chen, Daniel Israel +2
The premise and promise of KV (cache) eviction is simple: higher throughput can be achieved by evicting some entries from the KV cache, at a negligible cost to quality. This holds…
Probabilistic Programs of Thought
Poorva Garg, Renato Lui Geh, Daniel Israel +3
LLMs are widely used for code generation and mathematical reasoning tasks where they are required to generate structured output. They either need to reason about code, generate cod…
The Pitfalls of KV Cache Compression
Alex Chen, Renato Geh, Aditya Grover +2
KV cache compression promises increased throughput and efficiency with negligible loss in performance. While the gains in throughput are indisputable and recent literature has inde…
Adversarial Tokenization
Renato Lui Geh, Zilei Shao, Guy Van den Broeck
Current LLM pipelines account for only one possible tokenization for a given string, ignoring exponentially many alternative tokenizations during training and inference. For exampl…
Where is the signal in tokenization space?
Renato Lui Geh, Honghua Zhang, Kareem Ahmed +2
Large Language Models (LLMs) are typically shipped with tokenizers that deterministically encode text into so-called canonical token sequences, to which the LLMs assign probability…
dPASP: A Comprehensive Differentiable Probabilistic Answer Set Programming Environment For Neurosymbolic Learning and Reasoning
Renato Lui Geh, Jonas Gonçalves, Igor Cataneo Silveira +2
We present dPASP, a novel declarative probabilistic logic programming framework for differentiable neuro-symbolic reasoning. The framework allows for the specification of discrete…