3 citations · 3 across the 13 of their papers we have counts for
15 papers
LOCAL: Enabling Learning On-device Contiguously for Agent LLMs
Xinxin Liu, Jiaxin Li, Zibo Wang +7
On-device LLM agents interact repeatedly with users on local hardware, producing private traces that are valuable for adaptation but should not be sent to a remote trainer. Ideally…
Scheduling Mixed RL Rollouts Beyond Prefix Locality
Zetao Hong, Song Yuan, Yuanhao Ding +4
Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms. P…
MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training
Yikai Wang, Chuansai Zhou, Yuhang Zhou +10
Reinforcement learning (RL) post-training of large language models (LLMs) is computationally intensive and involves complex system pipelines with substantial debugging overhead. In…
Adaptive Matrix Multiplication for Dynamic Shapes on Ascend NPUs
Yuhang Zhou, Jiang Peng, Qianyu Jiang +7
Matrix Multiplication (MatMul) faces a "generalization crisis" driven by highly dynamic tensor shapes. This crisis is particularly acute on Ascend NPUs, where explicitly controlled…
RVANNS: Mixed-Precision Indexing and Locality-Aware Graph Traversal on RISC-V
Chengying Huan, Yudong Liu, Jianguo Wang +9
Approximate nearest neighbor search (ANNS) on CPUs is increasingly constrained by candidate-vector movement and decoding rather than peak arithmetic throughput. Although the RISC-V…
Bole: Efficient Tree Speculation for Hybrid-Attention Language Models
Li Wang, Yi Su, Xiabao Wu +9
Hybrid-attention large language models combine full attention with recurrent linear attention to reduce long-context inference costs, yet their autoregressive decoding remains memo…
TIDE-MC: Two-Sided Interpolative Decomposition for Billion-Scale GPU Matrix Completion
Chengying Huan, Yubo Wang, Pinhuan Wang +11
Matrix completion supports large-scale recommendation and scientific computing, yet existing GPU solvers commonly assume that the observed matrix or its dense factors fit in device…
SpecLA: Efficient Speculative Decoding for Linear-Attention Models
Zhibin Wang, Xuying Han, Zhaohua Yang +5
Linear-attention models replace the growing KV cache with recurrent states, but autoregressive decoding still reads, updates, and writes these states one token at a time. Speculati…
OrchANN: Hierarchical Orchestration for Skewed Out-of-Core Vector Search
Lizheng Chen, Pinhuan Wang, Shaonan Ma +9
At billion scale, approximate nearest neighbor search (ANNS) often becomes an out-of-core problem: the full vector collection and index structures exceed memory capacity, making qu…
Scaling Graph Chain-of-Thought Reasoning: A Multi-Agent Framework with Efficient LLM Serving
Chengying Huan, Ziheng Meng, Yongchao Liu +11
Graph Chain-of-Thought (Graph-CoT) enables large language models (LLMs) to perform step-by-step reasoning over graph-structured knowledge, but existing pipelines suffer from low ac…
ElasWave: An Elastic-Native System for Scalable Hybrid-Parallel Training
Xueze Kang, Guangyu Xiang, Yuxin Wang +16
Large-scale LLM pretraining now runs across -- accelerators, making failures routine and elasticity mandatory. We posit that an elastic-native training system must join…
STAR: Decode-Phase Rescheduling for LLM Inference
Zhibin Wang, Zetao Hong, Xue Li +8
Large Language Model (LLM) inference has emerged as a fundamental paradigm, however, variations in output length cause severe workload imbalance in the decode phase, particularly f…
Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
Zhibin Wang, Zhonghui Zhang, Yuhang Zhou +8
Recent advancements in Mixture of Experts (MoE) models have significantly increased their parameter scale as well as model performance. Extensive offloading techniques have been pr…
Chameleon: Adaptive Fault Tolerance for Distributed Training via Real-time Policy Selection
Yuhang Zhou, Zhibin Wang, Peng Jiang +12
Training large language models faces frequent interruptions due to various faults, demanding robust fault-tolerance. Existing backup-free methods, such as redundant computation, dy…
Understanding Large Language Models in Your Pockets: Performance Study on COTS Mobile Devices
Jie Xiao, Qianyi Huang, Xu Chen +1
As large language models (LLMs) increasingly integrate into every aspect of our work and daily lives, there are growing concerns about user privacy, which push the trend toward loc…