2 citations · 4 across the 35 of their papers we have counts for
36 papers · 1 filter
Streaming Video Editing with Easy Adaptation
Yujia Hu, Jiajun Li, Zihao He +1
In this paper, we propose SVEET, a framework that requires merely training on a pretrained bidirectional video diffusion model but supports high-quality streaming video editing in…
EDITBRIDGE: Towards Faithful and Efficient Ultra-High-Resolution Image Editing
Jiayi Song, Shijie Huang, Fangtai Wu +5
High-resolution image editing is increasingly demanded in professional workflows, yet existing diffusion-based models remain constrained to resolutions below 1K due to quadratic at…
Staying True to the Origin: Continuous Image Stylization with Smooth Transitions
Rui Xu, Hanmo Zhang, Songhua Liu
Recent advances in generative models have achieved remarkable performance in text- and image-conditioned editing. However, preserving the content of a given image while referencing…
Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration
Zihao He, Yunfeng Wu, Xinchao Wang +1
All-in-one image restoration seeks a single model that can recover images degraded by diverse and spatially non-uniform corruptions. However, many unified Transformers rely on fixe…
Flow-Map Distillation on Relation Manifolds for Image Restoration
Zihao He, Songhua Liu
Knowledge distillation for image restoration typically aligns intermediate features or relation matrices between teacher and student networks as static targets, ignoring the dynami…
LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation
Rongxiang Zhang, Songhua Liu
Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas rea…
Masked Generative Transformer Is What You Need for Image Editing
Wei Chow, Linfeng Li, Xian Sun +14
Diffusion models dominate image editing, yet their global denoising mechanism entangles edited regions with surrounding context, causing modifications to propagate into areas that…
Gated Condition Injection without Multimodal Attention: Towards Controllable Linear-Attention Transformers
Yuhe Liu, Zhenxiong Tan, Yujia Hu +2
Recent advances in diffusion-based controllable visual generation have led to remarkable improvements in image quality. However, these powerful models are typically deployed on clo…
ViBe: Ultra-High-Resolution Video Synthesis Born from Pure Images
Yunfeng Wu, Hongying Cheng, Zihao He +1
Transformer-based video diffusion models rely on 3D attention over spatial and temporal tokens, which incurs quadratic time and memory complexity and makes end-to-end training for…
ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformer
Ruonan Yu, Zhenxiong Tan, Zigeng Chen +2
Diffusion Transformers (DiTs) have demonstrated remarkable scalability and quality in image and video generation, prompting growing interest in extending them to controllable gener…
MultiAnimate: Pose-Guided Image Animation Made Extensible
Yingcheng Hu, Haowen Gong, Chuanguang Yang +3
Pose-guided human image animation aims to synthesize realistic videos of a reference character driven by a sequence of poses. While diffusion-based methods have achieved remarkable…
SpotEdit: Selective Region Editing in Diffusion Transformers
Zhibin Qin, Zhenxiong Tan, Zeqing Wang +2
Diffusion Transformer models have significantly advanced image editing by encoding conditional images and integrating them into transformer layers. However, most edits involve modi…
RunawayEvil: Jailbreaking the Image-to-Video Generative Models
Songping Wang, Rufan Qian, Yueming Lyu +5
Image-to-Video (I2V) generation synthesizes dynamic visual content from image and text inputs, providing significant creative control. However, the security of such multimodal syst…
CoDA: From Text-to-Image Diffusion Models to Training-Free Dataset Distillation
Letian Zhou, Songhua Liu, Xinchao Wang
Prevailing Dataset Distillation (DD) methods leveraging generative models confront two fundamental limitations. First, despite pioneering the use of diffusion models in DD and deli…
EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing
Wei Chow, Linfeng Li, Lingdong Kong +13
Recent advances in diffusion models (DMs) have achieved exceptional visual quality in image editing tasks. However, the global denoising dynamics of DMs inherently conflate local e…
FreeSwim: Revisiting Sliding-Window Attention Mechanisms for Training-Free Ultra-High-Resolution Video Generation
Yunfeng Wu, Jiayi Song, Zhenxiong Tan +2
The quadratic time and memory complexity of the attention mechanism in modern Transformer based video generators makes end-to-end training for ultra high resolution videos prohibit…
Vision Bridge Transformer at Scale
Zhenxiong Tan, Zeqing Wang, Xingyi Yang +2
We introduce Vision Bridge Transformer (ViBT), a large-scale instantiation of Brownian Bridge Models designed for conditional generation. Unlike traditional diffusion models that t…
SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skipping
Jiajun Li, Yue Ma, Xinyu Zhang +3
Recent studies on Visual Autoregressive (VAR) models have highlighted that high-frequency components, or later steps, in the generation process contribute disproportionately to inf…
Control and Realism: Best of Both Worlds in Layout-to-Image without Training
Bonan Li, Yinhan Hu, Songhua Liu +1
Layout-to-Image generation aims to create complex scenes with precise control over the placement and arrangement of subjects. Existing works have demonstrated that pre-trained Text…
Image Editing As Programs with Diffusion Models
Yujia Hu, Songhua Liu, Zhenxiong Tan +2
While diffusion models have achieved remarkable success in text-to-image generation, they encounter significant challenges with instruction-driven image editing. Our research highl…
Top-Down Compression: Revisit Efficient Vision Token Projection for Visual Instruction Tuning
Bonan li, Zicheng Zhang, Songhua Liu +2
Visual instruction tuning aims to enable large language models to comprehend the visual world, with a pivotal challenge lying in establishing an effective vision-to-language projec…
Flash Sculptor: Modular 3D Worlds from Objects
Yujia Hu, Songhua Liu, Xingyi Yang +1
Existing text-to-3D and image-to-3D models often struggle with complex scenes involving multiple objects and intricate interactions. Although some recent attempts have explored suc…
Ultra-Resolution Adaptation with Ease
Ruonan Yu, Songhua Liu, Zhenxiong Tan +1
Text-to-image diffusion models have achieved remarkable progress in recent years. However, training models for high-resolution image generation remains challenging, particularly wh…
OminiControl2: Efficient Conditioning for Diffusion Transformers
Zhenxiong Tan, Qiaochu Xue, Xingyi Yang +2
Fine-grained control of text-to-image diffusion transformer models (DiT) remains a critical challenge for practical deployment. While recent advances such as OminiControl and other…