11 papers
Improving LLM Reasoning via Dependency-Aware Query Decomposition and Logic-Parallel Content Expansion
Xianjun Gao, Jianchun Liu, Hongli Xu +1
The integration of Large Language Models (LLMs) into real-time Web applications, such as AI-powered search and conversational agents, presents a fundamental Web infrastructure chal…
Towards Communication-Efficient Decentralized Federated Graph Learning over Non-IID Data
Shilong Wang, Jianchun Liu, Hongli Xu +3
Decentralized Federated Graph Learning (DFGL) overcomes potential bottlenecks of the parameter server in FGL by establishing a peer-to-peer (P2P) communication network among worker…
Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
Jiaming Yan, Jianchun Liu, Hongli Xu +1
Mixture-of-Experts (MoE) has emerged as a promising architecture for modern large language models (LLMs). However, massive parameters impose heavy GPU memory (i.e., VRAM) demands,…
Adaptive KV-Cache Compression without Manually Setting Budget
Chenxia Tang, Jianchun Liu, Hongli Xu +1
Large language models (LLMs) inference relies heavily on KV-caches to accelerate autoregressive decoding, but the resulting memory footprint grows rapidly with sequence length, pos…
Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
Yujia Huo, Jianchun Liu, Hongli Xu +3
Federated fine-tuning (FedFT) of large language models (LLMs) has emerged as a promising solution for adapting models to distributed data environments while ensuring data privacy.…
FedQuad: Adaptive Layer-wise LoRA Deployment and Activation Quantization for Federated Fine-Tuning
Rukuo Li, Jianchun Liu, Hongli Xu +1
Federated fine-tuning (FedFT) provides an effective paradigm for fine-tuning large language models (LLMs) in privacy-sensitive scenarios. However, practical deployment remains chal…
Cross-region Model Training with Communication-Computation Overlapping and Delay Compensation
Ying Zhu, Yang Xu, Hongli Xu +3
Training large language models (LLMs) requires massive computational resources, often necessitating the aggregation of geographically distributed data centers (\ie, cross-region tr…
Collaborative Speculative Inference for Efficient LLM Inference Serving
Luyao Gao, Jianchun Liu, Hongli Xu +3
Speculative inference is a promising paradigm employing small speculative models (SSMs) as drafters to generate draft tokens, which are subsequently verified in parallel by the tar…
Accelerating End-Cloud Collaborative Inference via Near Bubble-free Pipeline Optimization
Luyao Gao, Jianchun Liu, Hongli Xu +3
End-cloud collaboration offers a promising strategy to enhance the Quality of Service (QoS) in DNN inference by offloading portions of the inference workload from end devices to cl…
Many Hands Make Light Work: Accelerating Edge Inference via Multi-Client Collaborative Caching
Wenyi Liang, Jianchun Liu, Hongli Xu +2
Edge inference is a technology that enables real-time data processing and analysis on clients near the data source. To ensure compliance with the Service-Level Objectives (SLOs), s…
ParallelSFL: A Novel Split Federated Learning Framework Tackling Heterogeneity Issues
Yunming Liao, Yang Xu, Hongli Xu +3
Mobile devices contribute more than half of the world's web traffic, providing massive and diverse data for powering various federated learning (FL) applications. In order to avoid…