Showing 2025Show all
2 papers · 1 filter
cs.CL2025
Role-aware Heuristic Episodic Attention for Conversational LLMs
Wanyang Hong, Zhaoning Zhang, Yi Chen +5
Large language models often lose track of persistent instructions and relevant information as multi-turn conversations grow. We study this cumulative contextual decay through three…
cs.DC2025
Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
Rui Li, Zhaoning Zhang, Libo Zhang +3
Speculative decoding (SD) accelerates LLM inference by verifying draft tokens in parallel. However, this method presents a critical trade-off: it improves throughput in low-load, m…