1 citations · 2 across the 5 of their papers we have counts for
7 papers
Reducing Latency of LLM Search Agent via Speculation-based Algorithm-System Co-Design
Zixiao Huang, Wen Zeng, Tianyu Fu +10
LLM-based search agents achieve strong performance but suffer from severe latency, as each step requires serialized LLM reasoning followed by action of tool execution. We revisit t…
RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformation
Chao Yu, Yuanqing Wang, Zhen Guo +26
Reinforcement learning (RL) has demonstrated immense potential in advancing artificial general intelligence, agentic intelligence, and embodied intelligence. However, the inherent…
STAlloc: Enhancing Memory Efficiency in Large-Scale Model Training with Spatio-Temporal Planning
Zixiao Huang, Junhao Hu, Hao Lin +9
The rapid scaling of large language models (LLMs) has significantly increased GPU memory pressure, which is further aggravated by training optimization techniques such as virtual p…
Efficient and Adaptable Overlapping for Computation and Communication via Signaling and Reordering
Ke Hong, Xiuhong Li, Minxu Liu +9
Generative models have achieved remarkable success across various applications, driving the demand for multi-GPU computing. Inter-GPU communication becomes a bottleneck in multi-GP…
Mixture of Attention Spans: Optimizing LLM Inference Efficiency with Heterogeneous Sliding-Window Lengths
Tianyu Fu, Haofeng Huang, Xuefei Ning +10
Sliding-window attention offers a hardware-efficient solution to the memory and throughput challenges of Large Language Models (LLMs) in long-context scenarios. Existing methods ty…
HETHUB: A Distributed Training System with Heterogeneous Cluster for Large-Scale Models
Si Xu, Zixiao Huang, Yan Zeng +11
Training large-scale models relies on a vast number of computing resources. For example, training the GPT-4 model (1.8 trillion parameters) requires 25000 A100 GPUs . It is a chall…