Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
How Does mHC Use Its Residual Streams? Selective Routing and Near-Identity Mixing
Pengxiang Zhao, Xing Li, Xianzhi Yu +2
Hyper-Connections and their manifold-constrained variant mHC widen a residual pathway from one stream to n, yet how trained models use this capacity remains unclear: how broadly bl…
cs.LG2026
AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
Yu-Yang Qian, Hao-Cong Wu, Chen Chen +4
Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified by the target model, has become a prevalent paradigm for acceleratin…
cs.LG2025
PreMoE: Proactive Inference for Efficient Mixture-of-Experts
Zehua Pei, Ying Zhang, Hui-Ling Zhen +6
Mixture-of-Experts (MoE) models offer dynamic computation, but are typically deployed as static full-capacity models, missing opportunities for deployment-specific specialization.…