4 papers
GMM: Generalized OS Memory Management for Programmable XPUs
Weixi Zhu, Bin Wang, Zhiyuan Fang +5
Since the AI boom, the fast evolving programmable XPU devices utilize high-bandwidth local memory to accelerate domain-specific tasks offloaded from the host CPU, resulting in a pr…
Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inference
Wenchen Han, Gingfung Matthew Yeung, Marco Barletta +3
Long-context inference is increasingly common in large language model (LLM) serving, driven by retrieval-augmented generation and agentic systems. In disaggregated inference, these…
Learning to Score: Tuning Cluster Schedulers through Reinforcement Learning
Martin Asenov, Qiwen Deng, Gingfung Yeung +1
Efficiently allocating incoming jobs to nodes in large-scale clusters can lead to substantial improvements in both cluster utilization and job performance. In order to allocate inc…
vPALs: Towards Verified Performance-aware Learning System For Resource Management
Guoliang He, Gingfung Yeung, Sheriffo Ceesay +1
Accurately predicting task performance at runtime in a cluster is advantageous for a resource management system to determine whether a task should be migrated due to performance de…