6 papers
The Thinking Leakage in Reinforcement Learning for LLMs
Wenxuan Zhu, Shuming Liu, Bang An +4
Hybrid reasoning models such as the Qwen3 series switch between a THINK mode and a NO-THINK mode through a few chat-template tokens. A popular recipe trains such models with reinfo…
HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models
Aznaur Aliev, Carlos Hinojosa, Abdelrahman Eldesokey +3
Safety alignment in large language models can be fragile under fine-tuning, as even benign task adaptation may increase harmful compliance. Existing defenses mainly follow two dire…
Defending Against Harmful Supervision Hidden in Benign Samples
Bang An, Yibo Yang, Dandan Guo +3
Existing defenses are effective when harmful content is explicitly mixed into downstream fine-tuning data, but crafted samples can instead hide harmful supervision inside benign ta…
Mitigating Model Merging Conflicts via Knowledge-Aware Task Vector Purification
Bang An, Yibo Yang, Philip Torr +1
Model merging aims to integrate task-specific abilities from individually fine-tuned models into a single model without extra training. In recent model merging methods, task vector…
Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence
Yibo Yang, Sihao Liu, Chuan Rao +5
Conventional low-rank adaptation methods build adapters without considering data context, leading to sub-optimal fine-tuning performance and severe forgetting of inherent world kno…
Talking Models: Distill Pre-trained Knowledge to Downstream Models via Interactive Communication
Zhe Zhao, Qingyun Liu, Huan Gui +3
Many recent breakthroughs in machine learning have been enabled by the pre-trained foundation models. By scaling up model parameters, training data, and computation resources, foun…