2 papers
cs.DC2025
SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding
Ziyi Zhang, Ziheng Jiang, Chengquan Jiang +5
Low-latency decoding for large language models (LLMs) is crucial for applications like chatbots and code assistants, yet generating long outputs remains slow in single-query settin…
cs.AR2024
UpDown: Programmable fine-grained Events for Scalable Performance on Irregular Applications
Andronicus Rajasukumar, Jiya Su, Yuqing +12
Applications with irregular data structures, data-dependent control flows and fine-grained data transfers (e.g., real-world graph computations) perform poorly on cache-based system…