7 papers
Epiphany-Aware KV Cache Eviction Without the Attention Matrix
Steven Kolawole, Virginia Smith
As reasoning models emit chains of thought tens of thousands of tokens long, KV cache increasingly becomes a deployment bottleneck. Existing cache eviction methods rank tokens by a…
What Survives When You Compress a Recursive Reasoner for the Edge?
Pearse Jim, Steven Kolawole, Opegbemi Matthias Busoye +2
Recursive reasoning models can solve complex structured tasks with only a few million parameters by repeatedly updating a latent state. Deploying these models on edge hardware requ…
PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systems
Nnaemeka Obiefuna, Samuel Oyeneye, Similoluwa Odunaiya +2
Privacy preserving machine learning deployments in sensitive deep learning applications; from medical imaging to autonomous systems; increasingly require combining multiple techniq…
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes
Steven Kolawole, Lucio Dery, Jean-François Kagy +3
Structured pruning is a promising approach to create smaller, faster large language models. However, existing methods typically rely on computing the gradient via backward passes,…
Semantic Agreement Enables Efficient Open-Ended LLM Cascades
Duncan Soiffer, Steven Kolawole, Virginia Smith
Cascade systems route computational requests to smaller models when possible and defer to larger models only when necessary, offering a promising approach to balance cost and quali…
PARALLELPROMPT: Extracting Parallelism from Large Language Model Queries
Steven Kolawole, Keshav Santhanam, Virginia Smith +1
LLM serving systems typically treat user prompts as monolithic inputs, optimizing inference through decoding tricks or inter-query batching. However, many real-world prompts contai…