21 citations · 105 across the 25 of their papers we have counts for
4 papers · 1 filter
SmartGen: Seamless Disaggregated LLM Inference with Selective KV Cache Transfer
Xuchuan Luo, Jiacheng Shen, Xin Wang +1
Disaggregating the prefill and decoding stages of large language model (LLM) inference into two separate sets of nodes is widely adopted in today's LLM serving systems. However, su…
Bringing Managed Language Support to WebAssembly with External Library Linking
Shuyao Jiang, Ruiying Zeng, Yangfan Zhou +1
WebAssembly (Wasm) has emerged as a powerful bytecode format for running applications with near-native performance in portable and secure environments. However, while Wasm currentl…
Debugging Performance Issues in WebAssembly Runtimes via Mutation-based Inference
Ruiying Zeng, Shuyao Jiang, Wenxuan Zhao +1
Performance debugging in WebAssembly (Wasm) runtimes is essential for ensuring the robustness of Wasm, especially since performance issues have frequently occurred in Wasm runtimes…
EdgeFlow: Fast Cold Starts for LLMs on Mobile Devices
Yongsheng Yan, Jiacheng Shen, Xuchuan Luo +1
Deploying large language models (LLMs) on mobile devices is an emerging trend to enable data privacy and offline accessibility of LLM applications. Modern mobile neural processing…