7 papers
Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agents
Yaoqi Chen, Haibin Lai, Yuru Feng +12
LLM-based agents increasingly tackle long-horizon tasks with interdependent decisions, where each action reshapes future constraints and intermediate errors can cascade. Existing R…
HyperParallel-MoE: Multi-Core Interleaved Scheduling for Fast MoE Training on Ascend NPUs
Zewen Jin, Congkun Ai, Guangpeng Zhang +7
Modern Mixture-of-Experts (MoE) models increasingly rely on large-scale AI accelerator clusters for efficient training. Ascend NPUs expose heterogeneous on-chip compute resources,…
Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
Long Zhao, Qinghe Wang, Jiaan Zhu +5
Reinforcement Learning from Human Feedback (RLHF) has become a key post-training paradigm for improving model quality. However, the synchronous three-stage RLHF pipeline is often b…
AdaCluster: Adaptive Query-Key Clustering for Sparse Attention in Video Generation
Haoyue Tan, Shengnan Wang, Yulin Qiao +5
Video diffusion transformers (DiTs) suffer from prohibitive inference latency due to quadratic attention complexity. Existing sparse attention methods either overlook semantic simi…
LiteCache: A Query Similarity-Driven, GPU-Centric KVCache Subsystem for Efficient LLM Inference
Jiawei Yi, Ping Gong, Youhui Bai +10
During LLM inference, KVCache memory usage grows linearly with sequence length and batch size and often exceeds GPU capacity. Recent proposals offload KV states to host memory and…
HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference
Ping Gong, Jiawei Yi, Shengnan Wang +13
Large Language Models (LLMs) have emerged as a pivotal research area, yet the attention module remains a critical bottleneck in LLM inference, even with techniques like KVCache to…