7 papers
Let the Carrier Carry the Attack: Preserving the Subject in Adversarial Image Generation
Linfeng Jiang, Steven McDonagh, Yuhang Chen +3
Strong unrestricted adversarial attacks can distort the primary object of an image, hereafter referred to as the subject. To preserve subject integrity without compromising attack…
Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Models
Xiaojie Gu, Guangxu Chen, Yuheng Yang +2
Large language models (LLMs) exhibit exceptional performance across various domains, yet they face critical safety concerns. Model editing has emerged as an effective approach to m…
Rethinking Inter-LoRA Orthogonality in Adapter Merging: Insights from Orthogonal Monte Carlo Dropout
Andi Zhang, Xuan Ding, Haofan Wang +2
We propose Orthogonal Monte Carlo Dropout, a mechanism that enforces strict orthogonality when combining sparse semantic vectors without extra time complexity. Low-Rank Adaptation…
Revoking Amnesia: RL-based Trajectory Optimization to Resurrect Erased Concepts in Diffusion Models
Daiheng Gao, Nanxiang Jiang, Andi Zhang +5
Concept erasure techniques have been widely deployed in T2I diffusion models to prevent inappropriate content generation for safety and copyright considerations. However, as models…
Concept-based Adversarial Attack: a Probabilistic Perspective
Andi Zhang, Xuan Ding, Steven McDonagh +1
We propose a concept-based adversarial attack framework that extends beyond single-image perturbations by adopting a probabilistic perspective. Rather than modifying a single image…
Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis
Boming Miao, Chunxiao Li, Xiaoxiao Wang +4
Diffusion models have achieved impressive success in generating photorealistic images, but challenges remain in ensuring precise semantic alignment with input prompts. Optimizing t…
LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models
Hossein Abdi, Mingfei Sun, Andi Zhang +2
Training large models with millions or even billions of parameters from scratch incurs substantial computational costs. Parameter Efficient Fine-Tuning (PEFT) methods, particularly…