SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models
arXiv:2404.06666 · doi:10.1145/3658644.3670295 10.1145/3658644.3670295 10.1145/3658644.3670295
Abstract
Text-to-image (T2I) models, such as Stable Diffusion, have exhibited remarkable performance in generating high-quality images from text descriptions in recent years. However, text-to-image models may be tricked into generating not-safe-for-work (NSFW) content, particularly in sexually explicit scenarios. Existing countermeasures mostly focus on filtering inappropriate inputs and outputs, or suppressing improper text embeddings, which can block sexually explicit content (e.g., naked) but may still be vulnerable to adversarial prompts -- inputs that appear innocent but are ill-intended. In this paper, we present SafeGen, a framework to mitigate sexual content generation by text-to-image models in a text-agnostic manner. The key idea is to eliminate explicit visual representations from the model regardless of the text input. In this way, the text-to-image model is resistant to adversarial prompts since such unsafe visual representations are obstructed from within. Extensive experiments conducted on four datasets and large-scale user studies demonstrate SafeGen's effectiveness in mitigating sexually explicit content generation while preserving the high-fidelity of benign images. SafeGen outperforms eight state-of-the-art baseline methods and achieves 99.4% sexual content removal performance. Furthermore, our constructed benchmark of adversarial prompts provides a basis for future development and evaluation of anti-NSFW-generation methods.
Accepted by ACM CCS 2024. Please cite this paper as "Xinfeng Li, Yuchen Yang, Jiangyi Deng, Chen Yan, Yanjiao Chen, Xiaoyu Ji, Wenyuan Xu. SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models. In Proceedings of ACM Conference on Computer and Communications Security (CCS), 2024."
References in corpus (14)
- Decoupled Weight Decay Regularization
- Learning Transferable Visual Models From Natural Language Supervision
- LAION-5B: An open large-scale dataset for training next generation image-text models
- GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- High-Resolution Image Synthesis with Latent Diffusion Models
- DiffWave: A Versatile Diffusion Model for Audio Synthesis
- DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
- Normalized and Geometry-Aware Self-Attention Network for Image Captioning
- On Aliased Resizing and Surprising Subtleties in GAN Evaluation
- SEGA: Instructing Text-to-Image Models using Semantic Guidance
- Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models
- SneakyPrompt: Jailbreaking Text-to-image Generative Models
- Circumventing Concept Erasure Methods For Text-to-Image Generative Models