4 citations · 4 across the 4 of their papers we have counts for
4 papers
DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training
Can Jin, Hongwu Peng, Mingcan Xiang +7
Sparse Mixture-of-Experts architectures are essential for scaling model capacity efficiently, yet the standard Top- routing imposes a rigid sparsity pattern that ignores the int…
Understanding and Alleviating Memory Consumption in RLHF for LLMs
Jin Zhou, Hanmei Yang, Steven +4
Fine-tuning with Reinforcement Learning with Human Feedback (RLHF) is essential for aligning large language models (LLMs). However, RLHF often encounters significant memory challen…
Scaler: Efficient and Effective Cross Flow Analysis
Steven, Tang, Mingcan Xiang +4
Performance analysis is challenging as different components (e.g.,different libraries, and applications) of a complex system can interact with each other. However, few existing too…
AdapMTL: Adaptive Pruning Framework for Multitask Learning Model
Mingcan Xiang, Steven Jiaxun Tang, Qizheng Yang +2
In the domain of multimedia and multimodal processing, the efficient handling of diverse data streams such as images, video, and sensor data is paramount. Model compression and mul…