15 papers
PAI-Actor: Cinematic Multi-Character Replacement in Dynamic Scenes
Bangxun Tang, Heyuan Gao, Yiren Song +4
We present PAI-Actor, a cinematic multi-character animation framework for character replacement in dynamic movie scenes. Unlike conventional animation systems that mainly drive a s…
EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing
Lan Chen, Qi Mao, Yiren Song +2
Visual-prompt-guided edit transfer aims to learn image transformations directly from example pairs, offering more precise and controllable editing than purely text-driven approache…
Mitty: Diffusion-based Human-to-Robot Video Generation
Yiren Song, Cheng Liu, Weijia Mao +1
Learning directly from human demonstration videos is a key milestone toward scalable and generalizable robot learning. Yet existing methods rely on intermediate representations suc…
IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learning
Yuanhang Li, Yiren Song, Junzhe Bai +4
We propose \textbf{IC-Effect}, an instruction-guided, DiT-based framework for few-shot video VFX editing that synthesizes complex effects (\eg flames, particles and cartoon charact…
H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videos
Hai Ci, Xiaokang Liu, Pei Yang +2
Robots that learn manipulation skills from everyday human videos could acquire broad capabilities without tedious robot data collection. We propose a video-to-video translation fra…
OmniPSD: Layered PSD Generation with Diffusion Transformer
Cheng Liu, Yiren Song, Haofan Wang +1
Recent advances in diffusion models have greatly improved image generation and editing, yet generating or reconstructing layered PSD files with transparent alpha channels remains h…
The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignment
Ziheng Ouyang, Yiren Song, Yaoli Liu +4
Previous works have explored various customized generation tasks given a reference image, but they still face limitations in generating consistent fine-grained details. In this pap…
OmniRefiner: Reinforcement-Guided Local Diffusion Refinement
Yaoli Liu, Ziheng Ouyang, Shengtao Lou +1
Reference-guided image generation has progressed rapidly, yet current diffusion models still struggle to preserve fine-grained visual details when refining a generated image using…
Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceivers
Yiqing Shi, Yiren Song, Mike Zheng Shou
Recent advances in diffusion transformers have shown remarkable generalization in visual synthesis, yet most dense perception methods still rely on text-to-image (T2I) generators d…
Personalized Vision via Visual In-Context Learning
Yuxin Jiang, Yuchao Gu, Yiren Song +2
Modern vision models, trained on large-scale annotated datasets, excel at predefined tasks but struggle with personalized vision -- tasks defined at test time by users with customi…
RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers
Yan Gong, Yiren Song, Yicheng Li +2
Inspired by the in-context learning mechanism of large language models (LLMs), a new paradigm of generalizable visual prompt-based image editing is emerging. Existing single-refere…
MCA-Bench: A Multimodal Benchmark for Evaluating CAPTCHA Robustness Against VLM-based Attacks
Zonglin Wu, Yule Xue, Yaoyao Feng +2
As automated attack techniques rapidly advance, CAPTCHAs remain a critical defense mechanism against malicious bots. However, existing CAPTCHA schemes encompass a diverse range of…
Autoregressive Images Watermarking through Lexical Biasing: An Approach Resistant to Regeneration Attack
Siqi Hui, Yiren Song, Sanping Zhou +3
Autoregressive (AR) image generation models have gained increasing attention for their breakthroughs in synthesis quality, highlighting the need for robust watermarking to prevent…
DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers
Zitong Wang, Hang Zhao, Qianyu Zhou +3
Diffusion models have recently motivated great success in many generation tasks like object removal. Nevertheless, existing image decomposition methods struggle to disentangle semi…
TransAnimate: Taming Layer Diffusion to Generate RGBA Video
Xuewei Chen, Zhimin Chen, Yiren Song
Text-to-video generative models have made remarkable advancements in recent years. However, generating RGBA videos with alpha channels for transparency and visual effects remains a…