37 citations · 150 across the 72 of their papers we have counts for
24 papers · 1 filter
MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising
Bingyuan Wang, Hengyu Meng, Zeyu Cai +4
Visual storytelling often uses nontypical aspect-ratio images like scroll paintings, comic strips, and panoramas to create an expressive and compelling narrative. While generative…
SPIRE: Semantic Prompt-Driven Image Restoration
Chenyang Qi, Zhengzhong Tu, Keren Ye +4
Text-driven diffusion models have become increasingly popular for various image editing tasks, including inpainting, stylization, and object replacement. However, it still remains…
HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation
Hongyu Liu, Xuan Wang, Ziyu Wan +4
This work presents HeadArtist for 3D head generation from text descriptions. With a landmark-guided ControlNet serving as the generative prior, we come up with an efficient pipelin…
Learning Naturally Aggregated Appearance for Efficient 3D Editing
Ka Leong Cheng, Qiuyu Wang, Zifan Shi +5
Neural radiance fields, which represent a 3D scene as a color field and a density field, have demonstrated great progress in novel view synthesis yet are unfavorable for editing du…
MagicStick: Controllable Video Editing via Control Handle Transformations
Yue Ma, Xiaodong Cun, Sen Liang +5
Text-based video editing has recently attracted considerable interest in changing the style or replacing the objects with a similar structure. Beyond this, we demonstrate that prop…
LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models
Qiang Wen, Zhefan Rao, Yazhou Xing +1
Enhancing a low-light noisy RAW image into a well-exposed and clean sRGB image is a significant challenge for modern digital cameras. Prior approaches have difficulties in recoveri…
Gaussian Shell Maps for Efficient 3D Human Generation
Rameen Abdal, Wang Yifan, Zifan Shi +6
Efficient generation of 3D digital humans is important in several industries, including virtual reality, social media, and cinematic production. 3D generative adversarial networks…
PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Driving
Zhili Chen, Maosheng Ye, Shuangjie Xu +2
We present a new interaction mechanism of prediction and planning for end-to-end autonomous driving, called PPAD (Iterative Interaction of Prediction and Planning Autonomous Drivin…
VideoCrafter1: Open Diffusion Models for High-Quality Video Generation
Haoxin Chen, Menghan Xia, Yingqing He +9
Video generation has increasingly gained interest in both academia and industry. Although commercial tools can generate plausible videos, there is a limited number of open-source m…
ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models
Yingqing He, Shaoshu Yang, Haoxin Chen +7
In this work, we investigate the capability of generating images from pre-trained diffusion models at much higher resolutions than the training image sizes. In addition, the genera…
ControlLLM: Augment Language Models with Tools by Searching on Graphs
Zhaoyang Liu, Zeqiang Lai, Zhangwei Gao +8
We present ControlLLM, a novel framework that enables large language models (LLMs) to utilize multi-modal tools for solving complex real-world tasks. Despite the remarkable perform…
Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection
Yazhou Xing, Amrita Mazumdar, Anjul Patney +5
Low dynamic range (LDR) cameras cannot deal with wide dynamic range inputs, frequently leading to local overexposure issues. We present a learning-based system to reduce these arti…
CoDeF: Content Deformation Fields for Temporally Consistent Video Processing
Hao Ouyang, Qiuyu Wang, Yuxi Xiao +6
We present the content deformation field CoDeF as a new type of video representation, which consists of a canonical content field aggregating the static contents in the entire vide…
Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation
Yingqing He, Menghan Xia, Haoxin Chen +8
Generating videos for visual storytelling can be a tedious and complex process that typically requires either live-action filming or graphics animation rendering. To bypass these c…
SAD: Segment Any RGBD
Jun Cen, Yizheng Wu, Kewei Wang +6
The Segment Anything Model (SAM) has demonstrated its effectiveness in segmenting any part of 2D RGB images. However, SAM exhibits a stronger emphasis on texture information while…
TextDiffuser: Diffusion Models as Text Painters
Jingye Chen, Yupan Huang, Tengchao Lv +3
Diffusion models have gained increasing attention for their impressive generation abilities but currently struggle with rendering accurate and coherent text. To address this issue,…
ERRA: An Embodied Representation and Reasoning Architecture for Long-horizon Language-conditioned Manipulation Tasks
Chao Zhao, Shuai Yuan, Chunli Jiang +4
This letter introduces ERRA, an embodied learning architecture that enables robots to jointly obtain three fundamental capabilities (reasoning, planning, and interaction) for solvi…
Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos
Yue Ma, Yingqing He, Xiaodong Cun +5
Generating text-editable and pose-controllable character videos have an imperious demand in creating various digital human. Nevertheless, this task has been restricted by the absen…
Rotating without Seeing: Towards In-hand Dexterity through Touch
Zhao-Heng Yin, Binghao Huang, Yuzhe Qin +2
Tactile information plays a critical role in human dexterity. It reveals useful contact information that may not be inferred directly from vision. In fact, humans can even perform…
Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition
Jun Cen, Shiwei Zhang, Xiang Wang +4
Open-set action recognition is to reject unknown human action cases which are out of the distribution of the training set. Existing methods mainly focus on learning better uncertai…
Human MotionFormer: Transferring Human Motions with Vision Transformers
Hongyu Liu, Xintong Han, Chengbin Jin +8
Human motion transfer aims to transfer motions from a target dynamic person to a source static one for motion synthesis. An accurate matching between the source person and the targ…
The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition
Jun Cen, Di Luan, Shiwei Zhang +5
Open-set Recognition (OSR) aims to identify test samples whose classes are not seen during the training process. Recently, Unified Open-set Recognition (UOSR) has been proposed to…
Learning 3D-aware Image Synthesis with Unknown Pose Distribution
Zifan Shi, Yujun Shen, Yinghao Xu +5
Existing methods for 3D-aware image synthesis largely depend on the 3D pose distribution pre-estimated on the training set. An inaccurate estimation may mislead the model into lear…
LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis
Jiapeng Zhu, Ceyuan Yang, Yujun Shen +4
This work presents an easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image. Establishing…