3 papers
cs.LG2026
SPRI: SVD-Partitioned Residual Initialization for Data-Constrained MoE Upcycling
Weiqiao Shan, Ruixiang Mao, Yuang Li +10
Mixture-of-Experts (MoE) models enable efficient scaling, but training them from scratch remains prohibitively expensive. MoE upcycling mitigates this cost by converting pretrained…
cs.AI2026
AdapShot: Adaptive Many-Shot In-Context Learning with Semantic-Aware KV Cache Reuse
Jie Ou, Jinyu Guo, Shiyao Guo +5
Many-Shot In-Context Learning (ICL) has emerged as a promising paradigm, leveraging extensive examples to unlock the reasoning potential of Large Language Models (LLMs). However, e…
cs.CL2026
ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs
Xunlei Chen, Jinyu Guo, Yuang Li +5
Large language models (LLMs) have advanced to encompass extensive knowledge across diverse domains. Yet controlling what a LLMs should not know is important for ensuring alignment…