9 citations · 34 across the 26 of their papers we have counts for
26 papers
JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation
Yinan Chen, Chuming Lin, Zhennan Chen +12
While instruction-based video editing has seen significant progress, joint audio-visual editing remains constrained by the absence of dedicated datasets and benchmarks. To bridge t…
PixVerve: Advancing Native UHR Image Generation to 100MP with a Large-Scale High-Quality Dataset
Haojun Chen, Haoyang He, Chengming Xu +11
Text-to-Image (T2I) models have recently seen notable progress around 1K and 2K resolution. With the extreme desire for better visual experience and the rapid development of imagin…
Soul: Breathe Life into Digital Human for High-fidelity Long-term Multimodal Animation
Jiangning Zhang, Junwei Zhu, Zhenye Gan +14
We propose a multimodal-driven framework for high-fidelity long-term digital human animation termed , which generates semantically coherent videos from a single-fram…
Transform Trained Transformer for Accelerating Native 4K Video Generation
Jiangning Zhang, Junwei Zhu, Teng Hu +9
Native 4K (21763840) video generation remains a critical challenge due to the quadratic computational explosion of full-attention as spatiotemporal resolution increases, ma…
DiP: Taming Diffusion Models in Pixel Space
Zhennan Chen, Junwei Zhu, Xu Chen +6
Diffusion models face a fundamental trade-off between generation quality and computational efficiency. Latent Diffusion Models (LDMs) offer an efficient solution but suffer from po…
FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning
Bo Yin, Xiaobin Hu, Xingyu Zhou +7
Diffusion models have achieved remarkable success in generative modeling, yet how to effectively adapt large pretrained models to new tasks remains challenging. We revisit the reco…
HumanVideo-MME: Benchmarking MLLMs for Human-Centric Video Understanding
Yuxuan Cai, Jiangning Zhang, Zhenye Gan +9
Multimodal Large Language Models (MLLMs) have demonstrated significant advances in visual understanding tasks involving both images and videos. However, their capacity to comprehen…
VTBench: Comprehensive Benchmark Suite Towards Real-World Virtual Try-on Models
Hu Xiaobin, Liang Yujie, Luo Donghao +5
While virtual try-on has achieved significant progress, evaluating these models towards real-world scenarios remains a challenge. A comprehensive benchmark is essential for three k…
SVFR: A Unified Framework for Generalized Video Face Restoration
Zhiyao Wang, Xu Chen, Chengming Xu +7
Face Restoration (FR) is a crucial area within image and video processing, focusing on reconstructing high-quality portraits from degraded inputs. Despite advancements in image FR,…
HiFiVFS: High Fidelity Video Face Swapping
Xu Chen, Keke He, Junwei Zhu +3
Face swapping aims to generate results that combine the identity from the source with attributes from the target. Existing methods primarily focus on image-based face swapping. Whe…
Sonic: Shifting Focus to Global Audio Perception in Portrait Animation
Xiaozhong Ji, Xiaobin Hu, Zhihong Xu +9
The study of talking face generation mainly explores the intricacies of synchronizing facial movements and crafting visually appealing, temporally-coherent animations. However, due…
MIMAFace: Face Animation via Motion-Identity Modulated Appearance Feature Learning
Yue Han, Junwei Zhu, Yuxiang Feng +5
Current diffusion-based face animation methods generally adopt a ReferenceNet (a copy of U-Net) and a large amount of curated self-acquired data to learn appearance features, as ro…
SVP: Style-Enhanced Vivid Portrait Talking Head Diffusion Model
Weipeng Tan, Chuming Lin, Chengming Xu +5
Talking Head Generation (THG), typically driven by audio, is an important and challenging task with broad application prospects in various fields such as digital humans, film produ…
ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians
Yufei Liu, Junshu Tang, Chu Zheng +4
High-fidelity 3D garment synthesis from text is desirable yet challenging for digital avatar creation. Recent diffusion-based approaches via Score Distillation Sampling (SDS) have…
RealTalk: Real-time and Realistic Audio-driven Face Generation with 3D Facial Prior-guided Identity Alignment Network
Xiaozhong Ji, Chuming Lin, Zhonggan Ding +6
Person-generic audio-driven face generation is a challenging task in computer vision. Previous methods have achieved remarkable progress in audio-visual synchronization, but there…
DF40: Toward Next-Generation Deepfake Detection
Zhiyuan Yan, Taiping Yao, Shen Chen +8
We propose a new comprehensive benchmark to revolutionize the current deepfake detection field to the next generation. Predominantly, existing works identify top-notch detection al…
Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control
Yue Han, Junwei Zhu, Keke He +7
Current face reenactment and swapping methods mainly rely on GAN frameworks, but recent focus has shifted to pre-trained diffusion models for their superior generation capabilities…
TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generation
Yufei Liu, Junwei Zhu, Junshu Tang +6
Texturing 3D humans with semantic UV maps remains a challenge due to the difficulty of acquiring reasonably unfolded UV. Despite recent text-to-3D advancements in supervising multi…
A Generalist FaceX via Learning Unified Facial Representation
Yue Han, Jiangning Zhang, Junwei Zhu +7
This work presents FaceX framework, a novel facial generalist model capable of handling diverse facial tasks simultaneously. To achieve this goal, we initially formulate a unified…
PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization
Xu Peng, Junwei Zhu, Boyuan Jiang +7
Recent advancements in personalized image generation using diffusion models have been noteworthy. However, existing methods suffer from inefficiencies due to the requirement for su…
High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learning
Chao Xu, Junwei Zhu, Jiangning Zhang +6
Recently, emotional talking face generation has received considerable attention. However, existing methods only adopt one-hot coding, image, or audio as emotion conditions, thus la…
Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator
Chao Xu, Shaoting Zhu, Junwei Zhu +4
Multimodal-driven talking face generation refers to animating a portrait with the given pose, expression, and gaze transferred from the driving image and video, or estimated from t…
SeedFormer: Patch Seeds based Point Cloud Completion with Upsample Transformer
Haoran Zhou, Yun Cao, Wenqing Chu +4
Point cloud completion has become increasingly popular among generation tasks of 3D point clouds, as it is a challenging yet indispensable problem to recover the complete shape of…
Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid
Wendong Zhang, Yunbo Wang, Bingbing Ni +1
Restoring reasonable and realistic content for arbitrary missing regions in images is an important yet challenging task. Although recent image inpainting models have made significa…