3 citations · 6 across the 5 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2024★ 1 cited
Boosting Deep Ensembles with Learning Rate Tuning
Hongpeng Jin, Yanzhao Wu
The Learning Rate (LR) has a high impact on deep learning training performance. A common practice is to train a Deep Neural Network (DNN) multiple times with different LR policies…
cs.LG2024★ 2 cited
DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Models
Maryam Akhavan Aghdam, Hongpeng Jin, Yanzhao Wu
Transformer-based Mixture-of-Experts (MoE) models have been driving several recent technological advancements in Natural Language Processing (NLP). These MoE models adopt a router…
cs.LG2023★ 3 cited
Rethinking Learning Rate Tuning in the Era of Large Language Models
Hongpeng Jin, Wenqi Wei, Xuyu Wang +2
Large Language Models (LLMs) represent the recent success of deep learning in achieving remarkable human-like predictive performance. It has become a mainstream strategy to leverag…