2 papers
cs.LG2026
KV Admission: Learning What to Write for Efficient Long-Context Inference
Yen-Chieh Huang, Pi-Cheng Hsiu, Rui Fang +1
Long-context LLM inference is bottlenecked by the quadratic attention complexity and linear KV cache growth. Prior approaches mitigate this via post-hoc selection or eviction but o…
cs.DC2025
Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism
Jinhui Wei, Ye Huang, Yuhui Zhou +3
In-situ LLM inference on end-user devices has gained significant interest due to its privacy benefits and reduced dependency on external infrastructure. However, as the decoding pr…