26 citations · 44 across the 14 of their papers we have counts for
14 papers
Audio-Visual Intelligence in Large Foundation Models
You Qin, Kai Liu, Shengqiong Wu +12
Audio-Visual Intelligence (AVI) has emerged as a central frontier in artificial intelligence, bridging auditory and visual modalities to enable machines that can perceive, generate…
AnimationBench: Are Video Models Good at Character-Centric Animation?
Leyi Wu, Pengjun Fang, Kai Sun +8
Video generation has advanced rapidly, with recent methods producing increasingly convincing animated results. However, existing benchmarks-largely designed for realistic videos-st…
AC-Foley: Reference-Audio-Guided Video-to-Audio Synthesis with Acoustic Transfer
Pengjun Fang, Yingqing He, Yazhou Xing +3
Existing video-to-audio (V2A) generation methods predominantly rely on text prompts alongside visual information to synthesize audio. However, two critical bottlenecks persist: sem…
SpA2V: Harnessing Spatial Auditory Cues for Audio-driven Spatially-aware Video Generation
Kien T. Pham, Yingqing He, Yazhou Xing +2
Audio-driven video generation aims to synthesize realistic videos that align with input audio recordings, akin to the human ability to visualize scenes from auditory input. However…
ModelGrow: Continual Text-to-Video Pre-training with Model Expansion and Language Understanding Enhancement
Zhefan Rao, Liya Ji, Yazhou Xing +6
Text-to-video (T2V) generation has gained significant attention recently. However, the costs of training a T2V model from scratch remain persistently high, and there is considerabl…
Large Motion Video Autoencoding with Cross-modal Video VAE
Yazhou Xing, Yang Fei, Yingqing He +4
Learning a robust video Variational Autoencoder (VAE) is essential for reducing video redundancy and facilitating efficient video generation. Directly applying image VAEs to indivi…
LLMs Meet Multimodal Generation and Editing: A Survey
Yingqing He, Zhaoyang Liu, Jingye Chen +13
With the recent advancement in large language models (LLMs), there is a growing interest in combining LLMs with multimodal learning. Previous surveys of multimodal large language m…
Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners
Yazhou Xing, Yingqing He, Zeyue Tian +2
Video and audio content creation serves as the core technique for the movie industry and professional users. Recently, existing diffusion-based methods tackle video and audio gener…
LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models
Qiang Wen, Zhefan Rao, Yazhou Xing +1
Enhancing a low-light noisy RAW image into a well-exposed and clean sRGB image is a significant challenge for modern digital cameras. Prior approaches have difficulties in recoveri…
Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection
Yazhou Xing, Amrita Mazumdar, Anjul Patney +5
Low dynamic range (LDR) cameras cannot deal with wide dynamic range inputs, frequently leading to local overexposure issues. We present a learning-based system to reduce these arti…
Deep Video Prior for Video Consistency and Propagation
Chenyang Lei, Yazhou Xing, Hao Ouyang +1
Applying an image processing algorithm independently to each video frame often leads to temporal inconsistency in the resulting video. To address this issue, we present a novel and…
Unsupervised Portrait Shadow Removal via Generative Priors
Yingqing He, Yazhou Xing, Tianjia Zhang +1
Portrait images often suffer from undesirable shadows cast by casual objects or even the face itself. While existing methods for portrait shadow removal require training on a large…
Invertible Image Signal Processing
Yazhou Xing, Zian Qian, Qifeng Chen
Unprocessed RAW data is a highly valuable image format for image editing and computer vision. However, since the file size of RAW data is huge, most users can only get access to pr…
Blind Video Temporal Consistency via Deep Video Prior
Chenyang Lei, Yazhou Xing, Qifeng Chen
Applying image processing algorithms independently to each video frame often leads to temporal inconsistency in the resulting video. To address this issue, we present a novel and g…