1 citations · 1 across the 7 of their papers we have counts for
8 papers
HERO-MoE: Historical Expert Routing with Scale-Preserving Fusion
Junxiang Qiu, Zhengsu Chen, Xinting Hu +6
Mixture-of-Experts (MoE) architectures have become a standard way to scale model capacity while keeping computation sparse, yet routing remains a key determinant of MoE quality and…
XPERT: Expert Knowledge Transfer for Effective Training of Language Models
Chang Liu, Boyu Shi, Xu Yang +1
Mixture-of-Experts (MoE) language models organize knowledge into explicitly routed expert modules, making expert-level representations traceable and analyzable. By analyzing expert…
Chain-based Distillation for Effective Initialization of Variable-Sized Small Language Models
Boyu Shi, YiCheng Jiang, Chang Liu +3
Large language models (LLMs) achieve strong performance but remain costly to deploy in resource-constrained settings. Training small language models (SLMs) from scratch is computat…
Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions
Boyu Shi, Chang Liu, ChuanBao Gao +2
Layer pruning efficiently reduces Large Language Model (LLM) computational costs but often triggers sudden performance collapse. Existing representation-based analyses struggle to…
Learngene Search Across Multiple Datasets for Building Variable-Sized Models
Boyu Shi, Junbo Zhou, Chang Liu +3
Deep learning methods are widely used under diverse resource constraints, resulting in models of varying sizes, such as the Vision Transformer (ViT) series. Deploying these models…
Building Variable-sized Models via Learngene Pool
Boyu Shi, Shiyu Xia, Xu Yang +3
Recently, Stitchable Neural Networks (SN-Net) is proposed to stitch some pre-trained networks for quickly building numerous networks with different complexity and performance trade…
RankMatch: A Novel Approach to Semi-Supervised Label Distribution Learning Leveraging Inter-label Correlations
Kouzhiqiang Yucheng Xie, Jing Wang, Yuheng Jia +2
This paper introduces RankMatch, an innovative approach for Semi-Supervised Label Distribution Learning (SSLDL). Addressing the challenge of limited labeled data, RankMatch effecti…
Exploiting Multi-Label Correlation in Label Distribution Learning
Zhiqiang Kou jing wang yuheng jia xin geng
Label Distribution Learning (LDL) is a novel machine learning paradigm that assigns label distribution to each instance. Many LDL methods proposed to leverage label correlation in…