21 citations · 30 across the 5 of their papers we have counts for
5 papers
UniMoMo: Expert Merging-Based MoE Acceleration for Large Recommendation Models
Lei Xin, Bin Gu, Peize Li +8
Sparse mixture-of-experts (MoE) layers expand recommendation capacity through conditional computation, yet a trained checkpoint still stores and routes over its full expert bank. W…
Training Report of TeleChat3-MoE
Xinzhang Liu, Chao Wang, Zhihao Yang +51
TeleChat3-MoE is the latest series of TeleChat large language models, featuring a Mixture-of-Experts (MoE) architecture with parameter counts ranging from 105 billion to over one t…
A Statistical Theory of Regularization-Based Continual Learning
Xuyang Zhao, Huiyuan Wang, Weiran Huang +1
We provide a statistical analysis of regularization-based continual learning on a sequence of linear regression tasks, with emphasis on how different regularization terms affect th…
ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations
Xuyang Zhao, Tianqi Du, Yisen Wang +2
Self-Supervised Learning (SSL) is a paradigm that leverages unlabeled data for model training. Empirical studies show that SSL can achieve promising performance in distribution shi…
Towards the Generalization of Contrastive Self-Supervised Learning
Weiran Huang, Mingyang Yi, Xuyang Zhao +1
Recently, self-supervised learning has attracted great attention, since it only requires unlabeled data for model training. Contrastive learning is one popular method for self-supe…