3 papers
cs.DC2025
NineToothed: A Triton-Based High-Level Domain-Specific Language for Machine Learning
Jiacheng Huang, Zimin Li, Yinghui Li +1
The emergence of deep learning domain-specific languages (DSLs) has substantially reduced the obstacles in developing high-performance, cross-platform compute kernels. However, cur…
cs.LG2025
SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models
Hang Wu, Jianian Zhu, Yinghui Li +3
Large Language Models (LLMs) present a critical trade-off between inference quality and computational cost: larger models offer superior capabilities but incur significant latency,…
cs.MM2025
FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
Jianian Zhu, Hang Wu, Haojie Wang +4
Multi-modal Large Language Models (MLLMs) serving systems commonly employ KV-cache compression to reduce memory footprint. However, existing compression methods introduce significa…