11 papers
Teacher-Guided Tool Learning for Small Language Models via Bidirectional KL Constraints
Haoxuan Gao, Qin-Wen Luo, Xiang Chen +5
External tools can extend the capabilities of small language models (SLMs), enabling their deployment in resource constrained environments. However, their limited capabilities make…
What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions
Yifeng Guo, Yanran Li, Xiang Chen
Sparse autoencoder (SAE) features are typically discovered from how their activations co-vary with behaviour, yet they are often used through interventions such as steering or abla…
Mind the Execution Gap: Action-Semantic Mismatch in World-Model Control
Shengtao Wen, Xiang Chen, Yu Tian +3
World-model controllers rely on action-conditioned dynamics for prediction and planning, yet real control systems often execute commands asynchronously due to communication delay,…
When Entropy Is Not Enough: Reclaiming Lost Semantics in LLM Output Length Prediction
Feiyang Ren, Shengtao Wen, Lingbing Guo +6
Efficient LLM serving is often bottlenecked by the need to pad sequences to a fixed maximum length, and this wastes compute and degrades throughput. Output length prediction enable…
VARM-Bench: Benchmarking Verifiable Structured Reasoning in Chinese Abusive Speech Moderation
Mingyu Yuan, Shengtao Wen, Lingbing Guo +2
The widespread circulation of abusive online content has increased the need for reliable moderation of Chinese social-media text. Existing Chinese benchmarks support label classifi…
Rethinking Normalization Placement for LLMs: Post-Norm under Curriculum Depth Growing
Sheng Ren, Yadong Wang, Naiqiang Tan +7
Pre-norm is the standard normalization placement in modern Transformers because it facilitates joint optimization of full-depth models. We ask whether this preference persists when…
Beyond Importance: Interchange-Sobol Sensitivity Reveals Task-Specific Content Channels in Transformer Components
Yifeng Guo, Jin-Hong Du, Xiang Chen
Mechanistic interpretability methods summarize a transformer component by a single importance score, conflating two distinct roles: a component may matter because it transports tas…
CoDA: Exploring Chain-of-Distribution Attacks and Post-Hoc Token-Space Repair for Medical Vision-Language Models
Xiang Chen, Fangfang Yang, Chunlei Meng +6
Medical vision--language models (MVLMs) are increasingly used as perceptual backbones in radiology pipelines and as the visual front end of multimodal assistants, yet their reliabi…
SafeThinker: Reasoning about Risk to Deepen Safety Beyond Shallow Alignment
Xianya Fang, Xianying Luo, Yadong Wang +8
Despite the intrinsic risk-awareness of Large Language Models (LLMs), current defenses often result in shallow safety alignment, rendering models vulnerable to disguised attacks (e…
Beyond Superficial Unlearning: Sharpness-Aware Robust Erasure of Hallucinations in Multimodal LLMs
Xianya Fang, Feiyang Ren, Xiang Chen +4
Multimodal LLMs are powerful but prone to object hallucinations, which describe non-existent entities and harm reliability. While recent unlearning methods attempt to mitigate this…
Rethinking Robust Adversarial Concept Erasure in Diffusion Models
Qinghong Yin, Yu Tian, Heming Yang +5
Concept erasure methods aim to remove specific unsafe target concepts in diffusion models while preserving image generation utility. To address the vulnerability that erased concep…