1 citations · 1 across the 6 of their papers we have counts for
6 papers
When Parallel Drafter Meets Parallel Speculative Decoding
Fuliang Liu, Xue Li, Kun Qian +4
DSpark-style parallel drafters have made speculative decoding highly effective, yet their draft phase remains serialized on the critical path of every round. Parallel speculative d…
DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference
Fuliang Liu, Xue Li, Ketai Zhao +7
Speculative decoding is an effective and lossless approach for accelerating LLM inference. However, existing widely adopted model-based draft designs, such as EAGLE3, improve accur…
Chordless Structure: A Pathway to Simple and Expressive GNNs
Hongxu Pan, Shuxian Hu, Mo Zhou +5
Researchers have proposed various methods of incorporating more structured information into the design of Graph Neural Networks (GNNs) to enhance their expressiveness. However, the…
CoDec: Prefix-Shared Decoding Kernel for LLMs
Zhibin Wang, Rui Ning, Chao Fang +12
Prefix-sharing among multiple prompts presents opportunities to combine the operations of the shared prefix, while attention computation in the decode stage, which becomes a critic…
Echo: Efficient Co-Scheduling of Hybrid Online-Offline Tasks for Large Language Model Serving
Zhibin Wang, Shipeng Li, Xue Li +7
Large language models have been widely deployed in various applications, encompassing both interactive online tasks and batched offline tasks. Given the burstiness and latency sens…
Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving
Zhibin Wang, Shipeng Li, Yuhang Zhou +7
User experience is a critical factor Large Language Model (LLM) serving systems must consider, where service level objectives (SLOs) considering the experience of individual reques…