5 papers · 1 filter
Decoupling Token Roles in Autoregressive Pretraining
Suqin Yuan, Runqi Lin, Kevin Qinghong Lin +4
Autoregressive pretraining increasingly draws on heterogeneous data, making it important to understand how a model learns from an individual token. The next-token prediction object…
Distinguishable Deletion: Unifying Knowledge Erasure and Refusal for Large Language Model Unlearning
Puning Yang, Junchi Yu, Qizhou Wang +3
Mitigating sensitive and harmful outputs is fundamental to ensuring safe deployment of LLMs. Existing approaches typically follow two paradigms: Knowledge Deletion (KD), which eras…
Safer by Diffusion, Broken by Context: Diffusion LLM's Safety Blessing and Its Failure Mode
Zeyuan He, Yupeng Chen, Lang Lin +7
Diffusion large language models (D-LLMs) offer an alternative to autoregressive LLMs (AR-LLMs) and have demonstrated advantages in generation efficiency. Beyond the utility benefit…
The Alignment Curse: Modality Alignment Supercharges Audio Attacks via Text Transfer
Yupeng Chen, Junchi Yu, Aoxi Liu +3
Recent advances in end-to-end trained omni-models have substantially improved audio capabilities by strengthening text-audio modality alignment. However, whether such alignment ina…
Causal Fine-Tuning under Latent Confounded Shift
Jialin Yu, Yuxiang Zhou, Haoxuan Li +6
Adapting to latent confounded shift remains a core challenge in modern AI. This setting is driven by hidden variables that induce spurious correlations between inputs and outputs d…