88 citations · 89 across the 3 of their papers we have counts for
3 papers
cs.LG2024★ 1 cited
CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition
Quang Pham, Giang Do, Huy Nguyen +8
Sparse mixture of experts (SMoE) offers an appealing solution to scale up the model complexity beyond the mean of increasing the network's depth or width. However, effective traini…
cs.LG2023
HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts
Giang Do, Khiem Le, Quang Pham +7
By routing input tokens to only a few split experts, Sparse Mixture-of-Experts has enabled efficient training of large language models. Recent findings suggest that fixing the rout…
cs.LG2022★ 88 cited
CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting
Gerald Woo, Chenghao Liu, Doyen Sahoo +2
Deep learning has been actively studied for time series forecasting, and the mainstream paradigm is based on the end-to-end training of neural network architectures, ranging from c…