2 papers
cs.AI2026
FastE: Readout-Triggered Token Compression for LLM Embedding Inference
Jinsong Shu, Jinyong Wen, Baokun Wang +4
In this study, we identify depth-dependent prefix redundancy in final-readout LLM embedding models, notably across representative backbones including Qwen3-Embedding and Qwen3-VL-E…
cs.AI2026
MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models
Jinsong Shu, Chenyang Wu, Zhongle Xie +2
Key-Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with context length and becomes a m…