Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
RoLaQ: Joint Rotation and Range Calibration for 2-Bit KV Caches in Multi-Head Latent Attention
Shiyu Ji, Peng Ling, Chen Lin +4
Multi-head latent attention (MLA) compresses the content of a key-value cache into a shared latent per token, making low-bit storage attractive for long-context inference. At two b…
cs.LG2025
Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decoding
Yixuan Wang, Yijun Liu, Shiyu ji +4
Large language models (LLMs) suffer from high inference latency due to the auto-regressive decoding process. Speculative decoding accelerates inference by generating multiple draft…
cs.LG2024
CRVQ: Channel-Relaxed Vector Quantization for Extreme Compression of LLMs
Yuzhuang Xu, Shiyu Ji, Qingfu Zhu +1
Powerful large language models (LLMs) are increasingly expected to be deployed with lower computational costs, enabling their capabilities on resource-constrained devices. Post-tra…