2 papers
cs.LG2025
AMS-QUANT: Adaptive Mantissa Sharing for Floating-point Quantization
Mengtao Lv, Ruiqi Zhu, Xinyu Wang +1
Large language models (LLMs) have demonstrated remarkable capabilities in various kinds of tasks, while the billion or even trillion parameters bring storage and efficiency bottlen…
cs.CL2025
PEARL: Parallel Speculative Decoding with Adaptive Draft Length
Tianyu Liu, Yun Li, Qitan Lv +4
Speculative decoding (SD), where an extra draft model is employed to provide multiple draft tokens first, and then the original target model verifies these tokens in parallel, has…