12 papers
Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse
Bole Ma, Jan Eitzinger, Harald Koestler +1
Multimodal agents repeatedly re-examine the same video frames, UI screenshots, and rendered artifacts as their context window slides and reasoning iterates, yet every look-back re-…
Move the Query, Not the Cache: Characterizing Cross-Instance Latent Attention Redistribution Across GPU Fabrics
Bole Ma, Jan Eitzinger, Harald Köstler +1
Frontier LLMs increasingly decide what a query attends to with a sparse-attention indexer that picks a few KV-cache blocks per query: attention's unit is now a small, reusable chun…
Leyline: KV Cache Directives for Agentic Inference
Bole Ma, Jan Eitzinger, Harald Koestler
Modern KV cache management assumes the chatbot workload: prompts arrive once and the cache grows append-only, so prefix caching and forward-only eviction are correct by constructio…
Diagnosing Overhead in Dispatch Operations: Cross-architecture Observatory
Bole Ma, Jan Eitzinger, Harald Koestler +1
AlltoAll dispatch is the dominant bottleneck of MoE expert parallelism, and the interconnect community has responded with four families of mitigations: predictive sample placement,…
The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures
Bole Ma, Ayesha Afzal, Jan Eitzinger +1
Power capping is the standard GPU energy lever in LLM serving, and it appears to work: throughput drops, power readings fall, and energy budgets are met. We show the appearance is…
Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Serving
Bole Ma, Jan Eitzinger, Harald Köstler
Agentic LLM workloads put bit-identical tokens at shifted positions every turn, voiding prefix caches at the first byte of divergence. Operators report cache-hit regressions rangin…