4 papers
Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference
Kexin Chu, Zecheng Lin, Dawei Xiang +7
Global KV-cache sharing is an effective optimization for accelerating large language model (LLM) inference, yet it introduces an API-visible timing side channel that lets adversari…
Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
Kexin Chu, Dawei Xiang, Zixu Shen +3
Mixture-of-Experts (MoE) has become a practical architecture for scaling LLM capacity while keeping per-token compute modest, but deploying MoE models on a single, memory-limited G…
ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
Zixu Shen, Kexin Chu, Yifan Zhang +3
The expansion of large language models is increasingly limited by the constrained memory capacity of modern GPUs. To mitigate this, Mixture-of-Experts (MoE) architectures activate…
PromptSculptor: Multi-Agent Based Text-to-Image Prompt Optimization
Dawei Xiang, Wenyan Xu, Kexin Chu +5
The rapid advancement of generative AI has democratized access to powerful tools such as Text-to-Image models. However, to generate high-quality images, users must still craft deta…