4 citations · 5 across the 19 of their papers we have counts for
4 papers · 1 filter
MESA: Improving MoE Safety Alignment via Decentralized Expertise
Yitong Sun, Yao Huang, Teng Li +5
Mixture-of-Experts (MoE) architectures scale Large Language Models (LLMs) efficiently, enabling greater capacity with reduced computational cost by dynamically routing inputs to re…
Mitigating Overthinking in Large Reasoning Models via Manifold Steering
Yao Huang, Huanran Chen, Shouwei Ruan +3
Recent advances in Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in solving complex tasks such as mathematics and coding. However, these models frequently…
Unveiling the Basin-Like Loss Landscape in Large Language Models
Huanran Chen, Yinpeng Dong, Zeming Wei +4
We discover the emergence of \textit{basins} in the loss landscape of large language models. As model scale increases, LLMs become progressively more resilient to random perturbati…
Scaling Laws for Black box Adversarial Attacks
Chuan Liu, Huanran Chen, Yichi Zhang +2
Adversarial examples exhibit cross-model transferability, enabling threatening black-box attacks on commercial models. Model ensembling, which attacks multiple surrogate models, is…