9 papers
DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptation
Héctor Laria, Yiping Han, Julian D. Santamaria +4
Adapting pre-trained text-to-image diffusion models, whether to learn new visual concepts or erase unwanted ones, is routinely evaluated on its intended effects alone. We argue thi…
Causal-Tune: Mining Causal Factors from Vision Foundation Models for Domain Generalized Semantic Segmentation
Yin Zhang, Yongqiang Zhang, Yaoyue Zheng +2
Fine-tuning Vision Foundation Models (VFMs) with a small number of parameters has shown remarkable performance in Domain Generalized Semantic Segmentation (DGSS). Most existing wor…
An h-space Based Adversarial Attack for Protection Against Few-shot Personalization
Xide Xu, Sandesh Kamath, Muhammad Atif Butt +1
The versatility of diffusion models in generating customized images from few samples raises significant privacy concerns, particularly regarding unauthorized modifications of priva…
Leveraging Semantic Attribute Binding for Free-Lunch Color Control in Diffusion Models
Héctor Laria, Alexandra Gomez-Villa, Jiang Qin +5
Recent advances in text-to-image (T2I) diffusion models have enabled remarkable control over various attributes, yet precise color specification remains a fundamental challenge. Ex…
Multi-label out-of-distribution detection via evidential learning
Eduardo Aguilar, Bogdan Raducanu, Petia Radeva
A crucial requirement for machine learning algorithms is not only to perform well, but also to show robustness and adaptability when encountering novel scenarios. One way to achiev…
Privacy Protection in Personalized Diffusion Models via Targeted Cross-Attention Adversarial Attack
Xide Xu, Muhammad Atif Butt, Sandesh Kamath +1
The growing demand for customized visual content has led to the rise of personalized text-to-image (T2I) diffusion models. Despite their remarkable potential, they pose significant…
Multi-Class Textual-Inversion Secretly Yields a Semantic-Agnostic Classifier
Kai Wang, Fei Yang, Bogdan Raducanu +1
With the advent of large pre-trained vision-language models such as CLIP, prompt learning methods aim to enhance the transferability of the CLIP model. They learn the prompt given…
Assessing Open-world Forgetting in Generative Image Model Customization
Héctor Laria, Alex Gomez-Villa, Kai Wang +2
Recent advances in diffusion models have significantly enhanced image generation capabilities. However, customizing these models with new classes often leads to unintended conseque…
The Expanding Scope of the Stability Gap: Unveiling its Presence in Joint Incremental Learning of Homogeneous Tasks
Sandesh Kamath, Albin Soutif-Cormerais, Joost van de Weijer +1
Recent research identified a temporary performance drop on previously learned tasks when transitioning to a new one. This drop is called the stability gap and has great consequence…