9 papers
Cosine Misleads: Auxiliary Losses Reshape Vision Language Models, Not Their Latents
XiuYu Zhang, Junfeng Fang, Zhenkai Liang
Latent visual reasoning (LVR) inserts supervised latent tokens between perception and answer generation in vision-language models (VLMs). The field uses alignment between these lat…
Self-Evaluation Is Already There: Eliciting Latent Judge Calibration in Base LLMs with Minimal Data
XiuYu Zhang, Yi Shan, Junfeng Fang +1
Large language models are increasingly evaluated by other models, raising a natural question: can a model predict how a judge will score its own output? We find that the ability is…
Do LLMs and VLMs Share Neurons for Inference? Evidence and Mechanisms of Cross-Modal Transfer
Chenhang Cui, An Zhang, Yuxin Chen +5
Large vision-language models (LVLMs) have rapidly advanced across various domains, yet they still lag behind strong text-only large language models (LLMs) on tasks that require mul…
AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraint
Leheng Sheng, Changshuo Shen, Weixiang Zhao +6
As LLMs are increasingly deployed in real-world applications, ensuring their ability to refuse malicious prompts, especially jailbreak attacks, is essential for safe and reliable u…
Self-Guard: Defending Large Reasoning Models via enhanced self-reflection
Jingnan Zheng, Jingjun Xu, Yanzhen Luo +6
The emergence of Large Reasoning Models (LRMs) introduces a new paradigm of explicit reasoning, enabling remarkable advances yet posing unique risks such as reasoning manipulation…
RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards
Jingnan Zheng, Xiangtian Ji, Yijun Lu +6
Large Language Models (LLMs) continue to exhibit vulnerabilities despite deliberate safety alignment efforts, posing significant risks to users and society. To safeguard against th…