Publications (18)
Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
Zhibin Wang, Zhonghui Zhang, Yuhang Zhou +8
Recent advancements in Mixture of Experts (MoE) models have significantly increased their parameter scale as well as model performance. Extensive offloading techniques have been pr…
A Composable Multimodal Framework for cine CMR-Text-Driven Prediction of Heart Failure Outcomes
Jianzhou Chen, Jinyang Sun, Xiumei Wang +6
Objective. Heart failure is one of the leading causes of death worldwide, with millions of deaths each year, according to data from the World Health Organization (WHO) and other pu…
STAR: Decode-Phase Rescheduling for LLM Inference
Zhibin Wang, Zetao Hong, Xue Li +8
Large Language Model (LLM) inference has emerged as a fundamental paradigm, however, variations in output length cause severe workload imbalance in the decode phase, particularly f…
Chordless Structure: A Pathway to Simple and Expressive GNNs
Hongxu Pan, Shuxian Hu, Mo Zhou +5
Researchers have proposed various methods of incorporating more structured information into the design of Graph Neural Networks (GNNs) to enhance their expressiveness. However, the…
Chameleon: Adaptive Fault Tolerance for Distributed Training via Real-time Policy Selection
Yuhang Zhou, Zhibin Wang, Peng Jiang +12
Training large language models faces frequent interruptions due to various faults, demanding robust fault-tolerance. Existing backup-free methods, such as redundant computation, dy…
Scaling Graph Chain-of-Thought Reasoning: A Multi-Agent Framework with Efficient LLM Serving
Chengying Huan, Ziheng Meng, Yongchao Liu +11
Graph Chain-of-Thought (Graph-CoT) enables large language models (LLMs) to perform step-by-step reasoning over graph-structured knowledge, but existing pipelines suffer from low ac…
CacheWeaver: Cache-Aware Evidence Ordering for Efficient Grounded RAG Inference
Kaizhen Tan, Rong Gu, Mingyuan Li
Retrieval-Augmented Generation (RAG) improves factual grounding, but it also lengthens prompts and raises prefill cost. Prefix caching in serving engines such as vLLM reduces this…
Hash Adaptive Bloom Filter
Rongbiao Xie, Meng Li, Zheyu Miao +4
Bloom filter is a compact memory-efficient probabilistic data structure supporting membership testing, i.e., to check whether an element is in a given set. However, as Bloom filter…
SmartSwap: Swap-Based Memory Optimization for LLM Training under Varying Operator Sequences
Zibo Wang, Yuhang Zhou, Zhibin Wang +13
The paper introduces SmartSwap, a swap‑based memory optimization framework that adapts to varying operator sequences in eager‑mode training of large language models, using a lightw…
Distributed Subgraph Enumeration via Backtracking-based Framework
Zhaokang Wang, Weiwei Hu, Chunfeng Yuan +2
Finding or monitoring subgraph instances that are isomorphic to a given pattern graph in a data graph is a fundamental query operation in many graph analytic applications, such as…
SpecLA: Efficient Speculative Decoding for Linear-Attention Models
Zhibin Wang, Xuying Han, Zhaohua Yang +5
Linear-attention models replace the growing KV cache with recurrent states, but autoregressive decoding still reads, updates, and writes these states one token at a time. Speculati…
Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving
Zhibin Wang, Shipeng Li, Yuhang Zhou +7
User experience is a critical factor Large Language Model (LLM) serving systems must consider, where service level objectives (SLOs) considering the experience of individual reques…
TIDE-MC: Two-Sided Interpolative Decomposition for Billion-Scale GPU Matrix Completion
Chengying Huan, Yubo Wang, Pinhuan Wang +11
Matrix completion supports large-scale recommendation and scientific computing, yet existing GPU solvers commonly assume that the observed matrix or its dense factors fit in device…
Echo: Efficient Co-Scheduling of Hybrid Online-Offline Tasks for Large Language Model Serving
Zhibin Wang, Shipeng Li, Xue Li +7
Large language models have been widely deployed in various applications, encompassing both interactive online tasks and batched offline tasks. Given the burstiness and latency sens…
CoDec: Prefix-Shared Decoding Kernel for LLMs
Zhibin Wang, Rui Ning, Chao Fang +12
Prefix-sharing among multiple prompts presents opportunities to combine the operations of the shared prefix, while attention computation in the decode stage, which becomes a critic…
Model Checking for Reinforcement Learning in Autonomous Driving: One Can Do More Than You Think!
Rong Gu
Most reinforcement learning (RL) platforms use high-level programming languages, such as OpenAI Gymnasium using Python. These frameworks provide various API and benchmarks for test…
CommonUppRoad: A Framework of Formal Modelling, Verifying, Learning, and Visualisation of Autonomous Vehicles
Rong Gu, Kaige Tan, Andreas Holck Høeg-Petersen +2
Combining machine learning and formal methods (FMs) provides a possible solution to overcome the safety issue of autonomous driving (AD) vehicles. However, there are gaps to be bri…
SSV: Sparse Speculative Verification for Efficient LLM Inference
Zhibin Wang, Ziyu Zhong, Nuo Shen +3
Speculative decoding and dynamic sparse attention are two complementary approaches for accelerating long-context LLM inference: the former amortizes target-model execution across m…