5 citations · 8 across the 4 of their papers we have counts for
4 papers · 1 filter
Fast LLM Post-training via Decoupled and Fastest-of-N Speculation
Rongxin Cheng, Kai Zhou, Xingda Wei +8
Rollout dominates the training time in large language model (LLM) post-training, where the trained model is used to generate tokens given a batch of prompts. This work, SpecActor,…
KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Serving
Rongxin Cheng, Yuxin Lai, Xingda Wei +2
Serving LLMs with a cluster of GPUs is common nowadays, where the serving system must meet strict latency SLOs required by applications. However, the stateful nature of LLM serving…
Characterizing the Dilemma of Performance and Index Size in Billion-Scale Vector Search and Breaking It with Second-Tier Memory
Rongxin Cheng, Yifan Peng, Xingda Wei +4
Vector searches on large-scale datasets are critical to modern online services like web search and RAG, which necessity storing the datasets and their index on the secondary storag…
Characterizing Off-path SmartNIC for Accelerating Distributed Systems
Xingda Wei, Rongxin Cheng, Yuhan Yang +2
SmartNICs have recently emerged as an appealing device for accelerating distributed systems. However, there has not been a comprehensive characterization of SmartNICs, and existing…