1 citations · 3 across the 16 of their papers we have counts for
19 papers
AC-Foley: Reference-Audio-Guided Video-to-Audio Synthesis with Acoustic Transfer
Pengjun Fang, Yingqing He, Yazhou Xing +3
Existing video-to-audio (V2A) generation methods predominantly rely on text prompts alongside visual information to synthesize audio. However, two critical bottlenecks persist: sem…
AlignVid: Training-Free Attention Scaling for Semantic Fidelity in Text-Guided Image-to-Video Generation
Yexin Liu, Wen-Jie Shu, Zile Huang +6
Text-guided image-to-video generation has made substantial progress, yet it still struggles to execute text-specified edits that require substantial changes to a reference image (\…
Zero-shot Synthetic Video Realism Enhancement via Structure-aware Denoising
Yifan Wang, Liya Ji, Zhanghan Ke +3
We propose an approach to enhancing synthetic video realism, which can re-render synthetic videos from a simulator in photorealistic fashion. Our realism enhancement approach is a…
Enhancing Diffusion-based Restoration Models via Difficulty-Adaptive Reinforcement Learning with IQA Reward
Xiaogang Xu, Ruihang Chu, Jian Wang +6
Reinforcement Learning (RL) has recently been incorporated into diffusion models, e.g., tasks such as text-to-image. However, directly applying existing RL methods to diffusion-bas…
CML-Bench: A Framework for Evaluating and Enhancing LLM-Powered Movie Scripts Generation
Mingzhe Zheng, Dingjie Song, Guanyu Zhou +7
Large Language Models (LLMs) have demonstrated remarkable proficiency in generating highly structured texts. However, while exhibiting a high degree of structural organization, mov…
TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head Synthesis
Shunian Chen, Hejin Huang, Yexin Liu +10
Audio-driven talking head synthesis has achieved remarkable photorealism, yet state-of-the-art (SOTA) models exhibit a critical failure: they lack generalization to the full spectr…
Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation
Harold Haodong Chen, Haojian Huang, Qifeng Chen +2
Recent advancements in video generation have enabled the creation of high-quality, visually compelling videos. However, generating videos that adhere to the laws of physics remains…
When Semantics Mislead Vision: Mitigating Large Multimodal Models Hallucinations in Scene Text Spotting and Understanding
Yan Shu, Hangui Lin, Yexin Liu +7
Large Multimodal Models (LMMs) have achieved impressive progress in visual perception and reasoning. However, when confronted with visually ambiguous or non-semantic scene text, th…
Model Reveals What to Cache: Profiling-Based Feature Reuse for Video Diffusion Models
Xuran Ma, Yexin Liu, Yaofu Liu +5
Recent advances in diffusion models have demonstrated remarkable capabilities in video generation. However, the computational intensity remains a significant challenge for practica…
VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention
Mingzhe Zheng, Yongqi Xu, Haojian Huang +8
Current video generation models excel at short clips but fail to produce cohesive multi-shot narratives due to disjointed visual dynamics and fractured storylines. Existing solutio…
Temporal Regularization Makes Your Video Generator Stronger
Harold Haodong Chen, Haojian Huang, Xianfeng Wu +5
Temporal quality is a critical aspect of video generation, as it ensures consistent motion and realistic dynamics across frames. However, achieving high temporal coherence and dive…
VideoMerge: Towards Training-free Long Video Generation
Siyang Zhang, Harry Yang, Ser-Nam Lim
Long video generation remains a challenging and compelling topic in computer vision. Diffusion based models, among the various approaches to video generation, have achieved state o…
LightGen: Efficient Image Generation through Knowledge Distillation and Direct Preference Optimization
Xianfeng Wu, Yajing Bai, Haoze Zheng +8
Recent advances in text-to-image generation have primarily relied on extensive datasets and parameter-heavy architectures. These requirements severely limit accessibility for resea…
Niagara: Normal-Integrated Geometric Affine Field for Scene Reconstruction from a Single View
Xianzu Wu, Zhenxin Ai, Harry Yang +3
Recent advances in single-view 3D scene reconstruction have highlighted the challenges in capturing fine geometric details and ensuring structural consistency, particularly in high…
Next Patch Prediction for Autoregressive Visual Generation
Yatian Pang, Peng Jin, Shuo Yang +8
Autoregressive models, built based on the Next Token Prediction (NTP) paradigm, show great potential in developing a unified framework that integrates both language and vision task…
VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention
Mingzhe Zheng, Yongqi Xu, Haojian Huang +8
Current video generation models excel at short clips but fail to produce cohesive multi-shot narratives due to disjointed visual dynamics and fractured storylines. Existing solutio…
Beyond Generation: Unlocking Universal Editing via Self-Supervised Fine-Tuning
Harold Haodong Chen, Harry Yang, Ser-Nam Lim
Recent advances in video generation have outpaced progress in video editing, which remains constrained by several limiting factors, namely: (a) the task's dependency on supervision…
DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses
Yatian Pang, Bin Zhu, Bin Lin +5
In this work, we present DreamDance, a novel method for animating human images using only skeleton pose sequences as conditional inputs. Existing approaches struggle with generatin…
Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly
Yexin Liu, Zhengyang Liang, Yueze Wang +8
Multimodal Large Language Models (MLLMs) have displayed remarkable performance in multi-modal tasks, particularly in visual comprehension. However, we reveal that MLLMs often gener…