46 citations · 55 across the 18 of their papers we have counts for
21 papers
TreeSpark: Learning to Draft the Trees You Verify for Speculative Decoding
Jihao Zhao, Qianyu Jiang, Shuaishuai Zu +6
Parallel drafters such as DFlash and DSpark reduce proposal latency, but early mismatches limit how much of each block can be accepted. Retaining alternatives introduces parent sta…
LOCAL: Enabling Learning On-device Contiguously for Agent LLMs
Xinxin Liu, Jiaxin Li, Zibo Wang +7
On-device LLM agents interact repeatedly with users on local hardware, producing private traces that are valuable for adaptation but should not be sent to a remote trainer. Ideally…
Scheduling Mixed RL Rollouts Beyond Prefix Locality
Zetao Hong, Song Yuan, Yuanhao Ding +4
Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms. P…
MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training
Yikai Wang, Chuansai Zhou, Yuhang Zhou +10
Reinforcement learning (RL) post-training of large language models (LLMs) is computationally intensive and involves complex system pipelines with substantial debugging overhead. In…
Adaptive Matrix Multiplication for Dynamic Shapes on Ascend NPUs
Yuhang Zhou, Jiang Peng, Qianyu Jiang +7
Matrix Multiplication (MatMul) faces a "generalization crisis" driven by highly dynamic tensor shapes. This crisis is particularly acute on Ascend NPUs, where explicitly controlled…
TIDE-MC: Two-Sided Interpolative Decomposition for Billion-Scale GPU Matrix Completion
Chengying Huan, Yubo Wang, Pinhuan Wang +11
Matrix completion supports large-scale recommendation and scientific computing, yet existing GPU solvers commonly assume that the observed matrix or its dense factors fit in device…
SpecLA: Efficient Speculative Decoding for Linear-Attention Models
Zhibin Wang, Xuying Han, Zhaohua Yang +5
Linear-attention models replace the growing KV cache with recurrent states, but autoregressive decoding still reads, updates, and writes these states one token at a time. Speculati…
SSV: Sparse Speculative Verification for Efficient LLM Inference
Zhibin Wang, Ziyu Zhong, Nuo Shen +3
Speculative decoding and dynamic sparse attention are two complementary approaches for accelerating long-context LLM inference: the former amortizes target-model execution across m…
DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference
Fuliang Liu, Xue Li, Ketai Zhao +7
Speculative decoding is an effective and lossless approach for accelerating LLM inference. However, existing widely adopted model-based draft designs, such as EAGLE3, improve accur…
STAR: Decode-Phase Rescheduling for LLM Inference
Zhibin Wang, Zetao Hong, Xue Li +8
Large Language Model (LLM) inference has emerged as a fundamental paradigm, however, variations in output length cause severe workload imbalance in the decode phase, particularly f…
SmartSwap: Swap-Based Memory Optimization for LLM Training under Varying Operator Sequences
Zibo Wang, Yuhang Zhou, Zhibin Wang +13
The increasing size of large language models (LLMs) has led to a surge in memory requirements during training, often exceeding the capacity of high-bandwidth memory (HBM). Swap-bas…
Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
Zhibin Wang, Zhonghui Zhang, Yuhang Zhou +8
Recent advancements in Mixture of Experts (MoE) models have significantly increased their parameter scale as well as model performance. Extensive offloading techniques have been pr…
Chameleon: Adaptive Fault Tolerance for Distributed Training via Real-time Policy Selection
Yuhang Zhou, Zhibin Wang, Peng Jiang +12
Training large language models faces frequent interruptions due to various faults, demanding robust fault-tolerance. Existing backup-free methods, such as redundant computation, dy…
Chordless Structure: A Pathway to Simple and Expressive GNNs
Hongxu Pan, Shuxian Hu, Mo Zhou +5
Researchers have proposed various methods of incorporating more structured information into the design of Graph Neural Networks (GNNs) to enhance their expressiveness. However, the…
CoDec: Prefix-Shared Decoding Kernel for LLMs
Zhibin Wang, Rui Ning, Chao Fang +12
Prefix-sharing among multiple prompts presents opportunities to combine the operations of the shared prefix, while attention computation in the decode stage, which becomes a critic…
Bingo: Radix-based Bias Factorization for Random Walk on Dynamic Graphs
Pinhuan Wang, Chengying Huan, Zhibin Wang +3
Random walks are a primary means for extracting information from large-scale graphs. While most real-world graphs are inherently dynamic, state-of-the-art random walk engines faile…
Echo: Efficient Co-Scheduling of Hybrid Online-Offline Tasks for Large Language Model Serving
Zhibin Wang, Shipeng Li, Xue Li +7
Large language models have been widely deployed in various applications, encompassing both interactive online tasks and batched offline tasks. Given the burstiness and latency sens…
Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving
Zhibin Wang, Shipeng Li, Yuhang Zhou +7
User experience is a critical factor Large Language Model (LLM) serving systems must consider, where service level objectives (SLOs) considering the experience of individual reques…
Unicron: Economizing Self-Healing LLM Training at Scale
Tao He, Xue Li, Zhibin Wang +4
Training large-scale language models is increasingly critical in various domains, but it is hindered by frequent failures, leading to significant time and economic costs. Current f…
GraphAr: An Efficient Storage Scheme for Graph Data in Data Lakes
Xue Li, Weibin Zeng, Zhibin Wang +4
Data lakes, increasingly adopted for their ability to store and analyze diverse types of data, commonly use columnar storage formats like Parquet and ORC for handling relational ta…
TRUST: Triangle Counting Reloaded on GPUs
Santosh Pandey, Zhibin Wang, Sheng Zhong +8
Triangle counting is a building block for a wide range of graph applications. Traditional wisdom suggests that i) hashing is not suitable for triangle counting, ii) edge-centric tr…