5 citations · 18 across the 9 of their papers we have counts for
10 papers
EchoPress: Query-Agnostic KV Cache Pruning via Virtual Context Reconstruction
Jiawei Lin, Saibo Geng, Thomas Bourgeat
KV cache pruning reduces long-context inference memory usage by evicting less important key-value pairs. KVzip estimates importance through context reconstruction: prompting a mode…
TRPrompt: Bootstrapping Query-Aware Prompt Optimization from Textual Rewards
Andreea Nica, Ivan Zakazov, Nicolas Mario Baldwin +2
Prompt optimization improves the reasoning abilities of large language models (LLMs) without requiring parameter updates to the target model. Following heuristic-based "Think step…
zip2zip: Inference-Time Adaptive Tokenization via Online Compression
Saibo Geng, Nathan Ranchin, Yunzhen yao +4
Tokenization efficiency plays a critical role in the performance and cost of large language models (LLMs), yet most models rely on static tokenizers optimized on general-purpose co…
JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models
Saibo Geng, Hudson Cooper, Michał Moskal +6
Reliably generating structured outputs has become a critical capability for modern language model (LM) applications. Constrained decoding has emerged as the dominant technology acr…
Byte BPE Tokenization as an Inverse string Homomorphism
Saibo Geng, Sankalp Gambhir, Chris Wendler +1
Tokenization is an important preprocessing step in the training and inference of large language models (LLMs). While there has been extensive research on the expressive power of th…
Sketch-Guided Constrained Decoding for Boosting Blackbox Large Language Models without Logit Access
Saibo Geng, Berkay Döner, Chris Wendler +2
Constrained decoding, a technique for enforcing constraints on language model outputs, offers a way to control text generation without retraining or architectural modifications. It…
Flows: Building Blocks of Reasoning and Collaborating AI
Martin Josifoski, Lars Klein, Maxime Peyrard +8
Recent advances in artificial intelligence (AI) have produced highly capable and controllable systems. This creates unprecedented opportunities for structured reasoning as well as…
Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning
Saibo Geng, Martin Josifoski, Maxime Peyrard +1
Despite their impressive performance, large language models (LMs) still struggle with reliably generating complex output structures when not finetuned to follow the required output…
Legal Transformer Models May Not Always Help
Saibo Geng, Rémi Lebret, Karl Aberer
Deep learning-based Natural Language Processing methods, especially transformers, have achieved impressive performance in the last few years. Applying those state-of-the-art NLP me…
An Enhanced MeanSum Method For Generating Hotel Multi-Review Summarizations
Saibo Geng, Diego Antognini
Multi-document summaritazion is the process of taking multiple texts as input and producing a short summary text based on the content of input texts. Up until recently, multi-docum…