4 papers
CacheBridge: Efficient Cross-Model KV Cache Transfer
Xingyu Qu, Siyuan Lu, Zhiyu Chen +2
Sharing context between LLMs in a multi-model system requires the receiving model to prefill the shared prefix because KV caches are model-specific. Recent closed-form cross-model…
Can Muon Fine-tune Adam-Pretrained Models?
Xingyu Qu, Peigeng Huang, Samuel Horvath
Muon has emerged as an efficient alternative to Adam for pretraining, yet remains underused for fine-tuning. A key obstacle is that most open models are pretrained with Adam, and n…
HAServe: Horizon-Aware Scheduling and Memory Co-Design for Efficient LLM Serving
Xingyu Qu, Tianhao Lin, Yiqi Li +2
Online large language model (LLM) services process requests that share context but differ in prefix structure and arrival timing. Finite capacity couples request admission and key-…
Vanishing Feature: Diagnosing Model Merging and Beyond
Xingyu Qu, Samuel Horvath
Model merging offers an efficient way to combine pre-trained neural networks but often suffers from inconsistent performance, especially when merging models with different initiali…