13 citations · 17 across the 7 of their papers we have counts for
7 papers
Analyzing Communication Predictability in LLM Training
Wenxue Li, Xiangzhou Liu, Yuxuan Li +9
Effective communication is essential in distributed training, with predictability being one of its most significant characteristics. However, existing studies primarily focus on ex…
DH-RAG: A Dynamic Historical Context-Powered Retrieval-Augmented Generation Method for Multi-Turn Dialogue
Feiyuan Zhang, Dezhi Zhu, James Ming +6
Retrieval-Augmented Generation (RAG) systems have shown substantial benefits in applications such as question answering and multi-turn dialogue \citep{lewis2020retrieval}. However,…
Unlocking Full Efficiency of Token Filtering in Large Language Model Training
Di Chai, Pengbo Li, Feiyuan Zhang +7
Token filtering has been proposed to enhance the utility of large language models (LLMs) by eliminating inconsequential tokens during training. While usingfewer tokens is expected…
MixNet: A Runtime Reconfigurable Optical-Electrical Fabric for Distributed Mixture-of-Experts Training
Xudong Liao, Yijun Sun, Han Tian +13
Mixture-of-Expert (MoE) models outperform conventional models by selectively activating different subnets, named experts, on a per-token basis. This gated computation generates dyn…
Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models
Yilun Jin, Zheng Li, Chenwei Zhang +19
Online shopping is a complex multi-task, few-shot learning problem with a wide and evolving range of entities, relations, and tasks. However, existing models and benchmarks are com…
Exploiting Student Parallelism for Efficient GPU Inference of BERT-like Models in Online Services
Weiyan Wang, Yilun Jin, Yiming Zhang +7
Due to high accuracy, BERT-like models have been widely adopted by text mining and web searching. However, large BERT-like models suffer from inefficient online inference, facing t…
PackVFL: Efficient HE Packing for Vertical Federated Learning
Liu Yang, Shuowei Cai, Di Chai +6
As an essential tool of secure distributed machine learning, vertical federated learning (VFL) based on homomorphic encryption (HE) suffers from severe efficiency problems due to d…