8 citations · 24 across the 13 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptation
Liangyu Wu, Qibin Liu, Alexander Yue +1
We present a lightweight approach to foundation modeling (\textbf{NEXUS}) that leverages pre-trained learning from collider physics data towards out-of-domain tasks in other scient…
cs.LG2024★ 2 cited
MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training
Pinxue Zhao, Hailin Zhang, Fangcheng Fu +9
Nowadays, Large Language Models (LLMs) have been trained using extended context lengths to foster more creative applications. However, long context training poses great challenges…
cs.LG2021★ 7 cited
EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate
Xiaonan Nie, Xupeng Miao, Shijie Cao +7
Mixture-of-experts (MoE) is becoming popular due to its success in improving the model quality, especially in Transformers. By routing tokens with a sparse gate to a few experts (i…