1 citations · 1 across the 3 of their papers we have counts for
7 papers
Revisiting Parameter Server in LLM Post-Training
Xinyi Wan, Penghui Qi, Guangxing Huang +3
Modern data parallel (DP) training favors collective communication over parameter servers (PS) for its simplicity and efficiency under balanced workloads. However, the balanced wor…
Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Caching
Chaoyi Ruan, Chao Bi, Kaiwen Zheng +3
Large Language Model (LLM) agents tackle data-intensive tasks such as deep research and code generation. However, their effectiveness depends on frequent interactions with knowledg…
ZeCO: Zero Communication Overhead Sequence Parallelism for Linear Attention
Yuhong Chou, Zehao Liu, Ruijie Zhu +6
Linear attention mechanisms deliver significant advantages for Large Language Models (LLMs) by providing linear computational complexity, enabling efficient processing of ultra-lon…
EMULATE: A Multi-Agent Framework for Determining the Veracity of Atomic Claims by Emulating Human Actions
Spencer Hong, Meng Luo, Xinyi Wan
Determining the veracity of atomic claims is an imperative component of many recently proposed fact-checking systems. Many approaches tackle this problem by first retrieving eviden…
PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
Xinyi Wan, Penghui Qi, Guangxing Huang +2
Pipeline parallelism (PP) is widely used for training large language models (LLMs), yet its scalability is often constrained by high activation memory consumption as the number of…
Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs
Longxu Dou, Qian Liu, Fan Zhou +38
Sailor2 is a family of cutting-edge multilingual language models for South-East Asian (SEA) languages, available in 1B, 8B, and 20B sizes to suit diverse applications. Building on…