6 papers
DeContext as Defense: Safe Image Editing in Diffusion Transformers
Linghui Shen, Mingyue Cui, Xingyi Yang
In-context diffusion models allow users to modify images with remarkable ease and realism. However, the same power raises serious privacy concerns: personal images can be easily ma…
Provably Robust Adaptation for Language-Empowered Foundation Models
Yuni Lai, Xiaoyu Xue, Linghui Shen +5
Language-empowered foundation models (LeFMs), such as CLIP and GraphCLIP, have transformed multimodal learning by aligning visual (or graph) features with textual representations,…
Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning
Liang Chen, Xueting Han, Li Shen +2
Harmful fine-tuning (HFT), performed directly on open-source LLMs or through Fine-tuning-as-a-Service, breaks safety alignment and poses significant threats. Existing methods aim t…
Exploration by Random Distribution Distillation
Zhirui Fang, Kai Yang, Jian Tao +4
Exploration remains a critical challenge in online reinforcement learning, as an agent must effectively explore unknown environments to achieve high returns. Currently, the main ex…
PEARL: Towards Permutation-Resilient LLMs
Liang Chen, Li Shen, Yang Deng +3
The in-context learning (ICL) capability of large language models (LLMs) enables them to perform challenging tasks using provided demonstrations. However, ICL is highly sensitive t…
Meta-TTT: A Meta-learning Minimax Framework For Test-Time Training
Chen Tao, Li Shen, Soumik Mondal
Test-time domain adaptation is a challenging task that aims to adapt a pre-trained model to limited, unlabeled target data during inference. Current methods that rely on self-super…