6 papers
PrefixFlow: Training-Time KV Caching for Schedule-Level Prefix Reuse in LLM RL Training
Pengbo Li, Feiyuan Zhang, Guangming Sheng +8
In reinforcement learning (RL) post-training, one prompt fans out into trajectories, yet standard trainers replay the shared prefix's forward and backward pass times; in lo…
UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Models
Guangxin He, Shen Nie, Fengqi Zhu +6
Diffusion LLMs have attracted growing interest, with plenty of recent work emphasizing their great potential in various downstream tasks; yet the long-context behavior of diffusion…
Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification
Tianyi Bai, Zengjie Hu, Fupeng Sun +7
Multi-modal large language models (MLLMs) have achieved remarkable capabilities by integrating visual perception with language understanding, enabling applications such as image-gr…
TAH-QUANT: Effective Activation Quantization in Pipeline Parallelism over Slow Network
Guangxin He, Yuan Cao, Yutong He +4
Decentralized training of large language models offers the opportunity to pool computational resources across geographically distributed participants, but is often bottlenecked by…
Zero-Indexing Internet Search Augmented Generation for Large Language Models
Guangxin He, Zonghong Dai, Jiangcheng Zhu +6
Retrieval augmented generation has emerged as an effective method to enhance large language model performance. This approach typically relies on an internal retrieval module that u…
TQA-Bench: Evaluating LLMs for Multi-Table Question Answering
Zipeng Qiu, Chenyue Li, You Peng +3
The advance of large language models (LLMs) has unlocked great opportunities in complex multi-modal data management tasks, particularly in question answering (QA) over complicated…