4 papers
FreeStory: Training-Free Character Consistency for Free-Form Visual Storytelling
Sibo Dong, Ismail Shaheen, Sarah Adel Bargal
Visual storytelling aims to generate image sequences that are both aligned with narrative prompts and consistent in character appearance across images. Recent training-free methods…
GIFT: Gradient-aware Immunization of diffusion models against malicious Fine-Tuning with safe concepts retention
Amro Abdalla, Ismail Shaheen, Dan DeGenaro +3
We present GIFT: a {G}radient-aware {I}mmunization technique to defend diffusion models against malicious {F}ine-{T}uning while preserving their ability to generate safe content. E…
ViSTA: Visual Storytelling using Multi-modal Adapters for Text-to-Image Diffusion Models
Sibo Dong, Ismail Shaheen, Maggie Shen +2
Text-to-image diffusion models have achieved remarkable success, yet generating coherent image sequences for visual storytelling remains challenging. A key challenge is effectively…
The Right Losses for the Right Gains: Improving the Semantic Consistency of Deep Text-to-Image Generation with Distribution-Sensitive Losses
Mahmoud Ahmed, Omer Moussa, Ismail Shaheen +5
One of the major challenges in training deep neural networks for text-to-image generation is the significant linguistic discrepancy between ground-truth captions of each image in m…