52 citations · 310 across the 37 of their papers we have counts for
37 papers
In-Domain GAN Inversion for Faithful Reconstruction and Editability
Jiapeng Zhu, Yujun Shen, Yinghao Xu +3
Generative Adversarial Networks (GANs) have significantly advanced image synthesis through mapping randomly sampled latent codes to high-fidelity synthesized images. However, apply…
RLIPv2: Fast Scaling of Relational Language-Image Pre-training
Hangjie Yuan, Shiwei Zhang, Xiang Wang +7
Relational Language-Image Pre-training (RLIP) aims to align vision representations with relational texts, thereby advancing the capability of relational reasoning in computer visio…
Regularized Mask Tuning: Uncovering Hidden Knowledge in Pre-trained Vision-Language Models
Kecheng Zheng, Wei Wu, Ruili Feng +6
Prompt tuning and adapter tuning have shown great potential in transferring pre-trained vision-language models (VLMs) to various downstream tasks. In this work, we design a new typ…
AnyDoor: Zero-shot Object-level Image Customization
Xi Chen, Lianghua Huang, Yu Liu +3
This work presents AnyDoor, a diffusion-based image generator with the power to teleport target objects to new scenes at user-specified locations in a harmonious way. Instead of tu…
MomentDiff: Generative Video Moment Retrieval from Random to Real
Pandeng Li, Chen-Wei Xie, Hongtao Xie +5
Video moment retrieval pursues an efficient and generalized solution to identify the specific temporal segments within an untrimmed video that correspond to a given language descri…
VideoComposer: Compositional Video Synthesis with Motion Controllability
Xiang Wang, Hangjie Yuan, Shiwei Zhang +6
The pursuit of controllability as a higher standard of visual content creation has yielded remarkable progress in customizable image synthesis. However, achieving controllable vide…
Lipschitz Singularities in Diffusion Models
Zhantao Yang, Ruili Feng, Han Zhang +8
Diffusion models, which employ stochastic differential equations to sample images through integrals, have emerged as a dominant class of generative models. However, the rationality…
Cones 2: Customizable Image Synthesis with Multiple Subjects
Zhiheng Liu, Yifei Zhang, Yujun Shen +7
Synthesizing images with user-specified subjects has received growing attention due to its practical applications. Despite the recent success in single subject customization, exist…
MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition
Xiang Wang, Shiwei Zhang, Zhiwu Qing +4
Current state-of-the-art approaches for few-shot action recognition achieve promising performance by conducting frame-level matching on learned visual features. However, they gener…
Scanning Only Once: An End-to-end Framework for Fast Temporal Grounding in Long Videos
Yulin Pan, Xiangteng He, Biao Gong +4
Video temporal grounding aims to pinpoint a video segment that matches the query description. Despite the recent advance in short-form videos (\textit{e.g.}, in minutes), temporal…
ViM: Vision Middleware for Unified Downstream Transferring
Yutong Feng, Biao Gong, Jianwen Jiang +4
Foundation models are pre-trained on massive data and transferred to downstream tasks via fine-tuning. This work presents Vision Middleware (ViM), a new learning paradigm that targ…
Cones: Concept Neurons in Diffusion Models for Customized Generation
Zhiheng Liu, Ruili Feng, Kai Zhu +6
Human brains respond to semantic features of presented stimuli with different neurons. It is then curious whether modern deep neural networks admit a similar behavior pattern. Spec…
CLIP-guided Prototype Modulating for Few-shot Action Recognition
Xiang Wang, Shiwei Zhang, Jun Cen +4
Learning from large-scale contrastive language-image pre-training like CLIP has shown remarkable success in a wide range of downstream tasks recently, but it is still under-explore…
VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation
Zhengxiong Luo, Dayou Chen, Yingya Zhang +6
A diffusion probabilistic model (DPM), which constructs a forward diffusion process by gradually adding noise to data points and learns the reverse denoising process to generate ne…
Rethinking Efficient Tuning Methods from a Unified Perspective
Zeyinzi Jiang, Chaojie Mao, Ziyuan Huang +3
Parameter-efficient transfer learning (PETL) based on large-scale pre-trained foundation models has achieved great success in various downstream applications. Existing tuning metho…
Composer: Creative and Controllable Image Synthesis with Composable Conditions
Lianghua Huang, Di Chen, Yu Liu +3
Recent large-scale generative models learned on big data are capable of synthesizing incredible images yet suffer from limited controllability. This work offers a new generation pa…
The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition
Jun Cen, Di Luan, Shiwei Zhang +5
Open-set Recognition (OSR) aims to identify test samples whose classes are not seen during the training process. Recently, Unified Open-set Recognition (UOSR) has been proposed to…
UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Training
Biao Gong, Shuai Tan, Yutong Feng +6
This work presents a unified knowledge protocol, called UKnow, which facilitates knowledge-based studies from the perspective of data. Particularly focusing on visual and linguisti…
LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis
Jiapeng Zhu, Ceyuan Yang, Yujun Shen +4
This work presents an easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image. Establishing…
Dimensionality-Varying Diffusion Process
Han Zhang, Ruili Feng, Zhantao Yang +7
Diffusion models, which learn to reverse a signal destruction process to generate new data, typically require the signal at each step to have the same dimension. We argue that, con…
Neural Dependencies Emerging from Learning Massive Categories
Ruili Feng, Kecheng Zheng, Kai Zhu +7
This work presents two astonishing findings on neural networks learned for large-scale image classification. 1) Given a well-trained model, the logits predicted for some category c…
Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator
Zifan Shi, Yinghao Xu, Yujun Shen +3
3D-aware image synthesis aims at learning a generative model that can render photo-realistic 2D images while capturing decent underlying 3D shapes. A popular solution is to adopt t…
Improving GANs with A Dynamic Discriminator
Ceyuan Yang, Yujun Shen, Yinghao Xu +3
Discriminator plays a vital role in training generative adversarial networks (GANs) via distinguishing real and synthesized samples. While the real data distribution remains the sa…
Rank Diminishing in Deep Neural Networks
Ruili Feng, Kecheng Zheng, Yukun Huang +3
The rank of neural networks measures information flowing across layers. It is an instance of a key structural condition that applies across broad domains of machine learning. In pa…