22 citations · 33 across the 23 of their papers we have counts for
8 papers · 1 filter
Hunyuan-A13B Technical Report
Tencent Hunyuan Team, Ao Liu, Botong Zhou +72
We present Hunyuan-A13B, an open-source large language model based on a Mixture-of-Experts architecture. It contains 80 billion total parameters but activates only 13 billion durin…
Dynamic Important Example Mining for Reinforcement Finetuning
Haoru Tan, Sitong Wu, Yanfeng Chen +9
Reinforcement fine-tuning (RFT) is increasingly used to strengthen the reasoning abilities of large models, yet its effectiveness is bound by how training data are selected and use…
Redesign Mixture-of-Experts Routers with Manifold Power Iteration
Songhao Wu, Ang Lv, Ruobing Xie +1
Router is the cornerstone component to the Mixture-of-Experts models. Serving as expert proxies, the rows of the router matrix compute their similarity to the MoE inputs to determi…
Hybrid Policy Distillation for LLMs
Wenhong Zhu, Ruobing Xie, Rui Wang +1
Knowledge distillation (KD) is a powerful paradigm for compressing large language models (LLMs), whose effectiveness depends on intertwined choices of divergence direction, optimiz…
OMNIFLOW: A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning
Hao Wu, Yongheng Zhang, Yuan Gao +7
Large Language Models (LLMs) have demonstrated exceptional logical reasoning capabilities but frequently struggle with the continuous spatiotemporal dynamics governed by Partial Di…
Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
Wenkai Yang, Weijie Liu, Ruobing Xie +3
On-policy distillation (OPD), which aligns the student with the teacher's logit distribution on student-generated trajectories, has demonstrated strong empirical gains in improving…
DQuant: Accurate Low-bit Post-Training Weight Quantization for LLMs
Xianglong Yan, ChengZhu Bao, Zhiteng Li +5
Large language models (LLMs) deliver strong performance, but their high compute and memory costs make deployment difficult in resource-constrained scenarios. Weight-only post-train…
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models
Hengyuan Zhang, Zhihao Zhang, Mingyang Wang +26
Mechanistic Interpretability (MI) has emerged as a vital approach to demystify the opaque decision-making of Large Language Models (LLMs). However, existing reviews primarily treat…