349 citations · 1.6k across the 67 of their papers we have counts for
68 papers
MagicEdit: High-Fidelity and Temporally Coherent Video Editing
Jun Hao Liew, Hanshu Yan, Jianfeng Zhang +2
In this report, we present MagicEdit, a surprisingly simple yet effective solution to the text-guided video editing task. We found that high-fidelity and temporally coherent video-…
MagicAvatar: Multimodal Avatar Generation and Animation
Jianfeng Zhang, Hanshu Yan, Zhongcong Xu +2
This report presents MagicAvatar, a framework for multimodal video generation and animation of human avatars. Unlike most existing methods that generate avatar-centric videos direc…
Dataset Quantization
Daquan Zhou, Kai Wang, Jianyang Gu +5
State-of-the-art deep neural networks are trained with large amounts (millions or even billions) of data. The expensive computation and memory costs make it difficult to train them…
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs
Yang Zhao, Zhijie Lin, Daquan Zhou +3
LLMs have demonstrated remarkable abilities at interacting with humans through language, especially with the usage of instruction-following data. Recent advancements in LLMs, such…
AdjointDPM: Adjoint Sensitivity Method for Gradient Backpropagation of Diffusion Probabilistic Models
Jiachun Pan, Jun Hao Liew, Vincent Y. F. Tan +2
Existing customization methods require access to multiple reference examples to align pre-trained diffusion probabilistic models (DPMs) with user-provided concepts. This paper aims…
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model
Sihan Chen, Xingjian He, Handong Li +3
Due to the limited scale and quality of video-text training corpus, most vision-language foundation models employ image-text datasets for pretraining and primarily focus on modelin…
Delving Deeper into Data Scaling in Masked Image Modeling
Cheng-Ze Lu, Xiaojie Jin, Qibin Hou +3
Understanding whether self-supervised learning methods can scale with unlimited data is crucial for training large-scale models. In this work, we conduct an empirical study on the…
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending
Xingjian He, Sihan Chen, Fan Ma +7
Large-scale image-text contrastive pre-training models, such as CLIP, have been demonstrated to effectively learn high-quality multimodal representations. However, there is limited…
OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis
Hongyi Xu, Guoxian Song, Zihang Jiang +6
We present OmniAvatar, a novel geometry-guided 3D head synthesis model trained from in-the-wild unstructured images that is capable of synthesizing diverse identity-preserved 3D he…
AgileGAN3D: Few-Shot 3D Portrait Stylization by Augmented Transfer Learning
Guoxian Song, Hongyi Xu, Jing Liu +7
While substantial progresses have been made in automated 2D portrait stylization, admirable 3D portrait stylization from a single user photo remains to be an unresolved challenge.…
TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervision
Jiacheng Wei, Hao Wang, Jiashi Feng +2
In this paper, we investigate an open research task of generating controllable 3D textured shapes from the given textual descriptions. Previous works either require ground truth ca…
CMAE-V: Contrastive Masked Autoencoders for Video Action Recognition
Cheng-Ze Lu, Xiaojie Jin, Zhicheng Huang +3
Contrastive Masked Autoencoder (CMAE), as a new self-supervised framework, has shown its potential of learning expressive feature representations in visual image recognition. This…
PV3D: A 3D Generative Model for Portrait Video Generation
Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew +4
Recent advances in generative adversarial networks (GANs) have demonstrated the capabilities of generating stunning photo-realistic portrait images. While some prior works have app…
Diffusion Probabilistic Model Made Slim
Xingyi Yang, Daquan Zhou, Jiashi Feng +1
Despite the recent visually-pleasing results achieved, the massive computational cost has been a long-standing flaw for diffusion probabilistic models (DPMs), which, in turn, great…
AvatarGen: A 3D Generative Model for Animatable Human Avatars
Jianfeng Zhang, Zihang Jiang, Dingdong Yang +6
Unsupervised generation of 3D-aware clothed humans with various appearances and controllable geometries is important for creating virtual human avatars and other AR/VR applications…
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition
Qibin Hou, Cheng-Ze Lu, Ming-Ming Cheng +1
This paper does not attempt to design a state-of-the-art method for visual recognition but investigates a more efficient way to make use of convolutions to encode spatial features.…
Expanding Small-Scale Datasets with Guided Imagination
Yifan Zhang, Daquan Zhou, Bryan Hooi +2
The power of DNNs relies heavily on the quantity and quality of training data. However, collecting and annotating data on a large scale is often expensive and time-consuming. To ad…
MagicVideo: Efficient Video Generation With Latent Diffusion Models
Daquan Zhou, Weimin Wang, Hanshu Yan +3
We present an efficient text-to-video generation framework based on latent diffusion models, termed MagicVideo. MagicVideo can generate smooth video clips that are concordant with…
MagicMix: Semantic Mixing with Diffusion Models
Jun Hao Liew, Hanshu Yan, Daquan Zhou +1
Have you ever imagined what a corgi-alike coffee machine or a tiger-alike rabbit would look like? In this work, we attempt to answer these questions by exploring a new task called…
Reachability-Aware Laplacian Representation in Reinforcement Learning
Kaixin Wang, Kuangqi Zhou, Jiashi Feng +2
In Reinforcement Learning (RL), Laplacian Representation (LapRep) is a task-agnostic state representation that encodes the geometry of the environment. A desirable property of LapR…
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning
Dongze Lian, Daquan Zhou, Jiashi Feng +1
Existing fine-tuning methods either tune all parameters of the pre-trained model (full fine-tuning), which is not efficient, or only tune the last linear layer (linear probing), wh…
AvatarGen: a 3D Generative Model for Animatable Human Avatars
Jianfeng Zhang, Zihang Jiang, Dingdong Yang +6
Unsupervised generation of clothed virtual humans with various appearance and animatable poses is important for creating 3D human avatars and other AR/VR applications. Existing met…
Contrastive Masked Autoencoders are Stronger Vision Learners
Zhicheng Huang, Xiaojie Jin, Chengze Lu +5
Masked image modeling (MIM) has achieved promising results on various vision tasks. However, the limited discriminability of learned representation manifests there is still plenty…
Clover: Towards A Unified Video-Language Alignment and Fusion Model
Jingjia Huang, Yinan Li, Jiashi Feng +3
Building a universal Video-Language model for solving various video understanding tasks (\emph{e.g.}, text-video retrieval, video question answering) is an open challenge to the ma…