32 citations · 52 across the 3 of their papers we have counts for
3 papers
cs.LG2023★ 2 cited
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
Ranggi Hwang, Jianyu Wei, Shijie Cao +4
Large language models (LLMs) based on transformers have made significant strides in recent years, the success of which is driven by scaling up their model size. Despite their high…
cs.DC2022★ 32 cited
Tutel: Adaptive Mixture-of-Experts at Scale
Changho Hwang, Wei Cui, Yifan Xiong +12
Sparsely-gated mixture-of-experts (MoE) has been widely adopted to scale deep learning models to trillion-plus parameters with fixed computational cost. The algorithmic performance…
cs.LG2017★ 18 cited
Confident Multiple Choice Learning
Kimin Lee, Changho Hwang, KyoungSoo Park +1
Ensemble methods are arguably the most trustworthy techniques for boosting the performance of machine learning models. Popular independent ensembles (IE) relying on naive averaging…