64 citations · 292 across the 46 of their papers we have counts for
20 papers · 1 filter
DreamVideo: High-Fidelity Image-to-Video Generation with Image Retention and Text Guidance
Cong Wang, Jiaxi Gu, Panwen Hu +3
Image-to-video generation, which aims to generate a video starting from a given reference image, has drawn great attention. Existing methods try to extend pre-trained text-guided i…
Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Images
Cuican Yu, Guansong Lu, Yihan Zeng +7
Generating 3D faces from textual descriptions has a multitude of applications, such as gaming, movie, and robotics. Recent progresses have demonstrated the success of unconditional…
GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training
Xinchi Deng, Han Shi, Runhui Huang +7
Cross-modal pre-training has shown impressive performance on a wide range of downstream tasks, benefiting from massive image-text pairs collected from the Internet. In practice, on…
DiffCloth: Diffusion Based Garment Synthesis and Manipulation via Structural Cross-modal Semantic Alignment
Xujie Zhang, Binbin Yang, Michael C. Kampffmeyer +6
Cross-modal garment synthesis and manipulation will significantly benefit the way fashion designers generate garments and modify their designs via flexible linguistic interfaces.Cu…
DiffDis: Empowering Generative Diffusion Model with Cross-Modal Discrimination Capability
Runhui Huang, Jianhua Han, Guansong Lu +4
Recently, large-scale diffusion models, e.g., Stable diffusion and DallE2, have shown remarkable results on image synthesis. On the other hand, large-scale cross-modal pre-trained…
MixReorg: Cross-Modal Mixed Patch Reorganization is a Good Mask Learner for Open-World Semantic Segmentation
Kaixin Cai, Pengzhen Ren, Yi Zhu +5
Recently, semantic segmentation models trained with image-level text supervision have shown promising results in challenging open-world scenarios. However, these models still face…
FULLER: Unified Multi-modality Multi-task 3D Perception via Multi-level Gradient Calibration
Zhijian Huang, Sihao Lin, Guiyu Liu +5
Multi-modality fusion and multi-task learning are becoming trendy in 3D autonomous driving scenario, considering robust prediction and computation budget. However, naively extendin…
Fashion Matrix: Editing Photos by Just Talking
Zheng Chong, Xujie Zhang, Fuwei Zhao +2
The utilization of Large Language Models (LLMs) for the construction of AI systems has garnered significant attention across diverse fields. The extension of LLMs to the domain of…
Surfer: Progressive Reasoning with World Models for Robotic Manipulation
Pengzhen Ren, Kaidong Zhang, Hetao Zheng +5
Considering how to make the model accurately understand and follow natural language instructions and perform actions consistent with world knowledge is a key challenge in robot man…
CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation
Xiwen Liang, Liang Ma, Shanshan Guo +4
Understanding and following natural language instructions while navigating through complex, real-world environments poses a significant challenge for general-purpose robots. These…
UniDiff: Advancing Vision-Language Models with Generative and Discriminative Learning
Xiao Dong, Runhui Huang, Xiaoyong Wei +4
Recent advances in vision-language pre-training have enabled machines to perform better in multimodal object discrimination (e.g., image-text semantic alignment) and image synthesi…
RealignDiff: Boosting Text-to-Image Diffusion Model with Coarse-to-fine Semantic Re-alignment
Zutao Jiang, Guian Fang, Jianhua Han +5
Recent advances in text-to-image diffusion models have achieved remarkable success in generating high-quality, realistic images from textual descriptions. However, these approaches…
Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data
Haonan Wang, Minbin Huang, Runhui Huang +7
Contrastive Language-Image Pre-training (CLIP) has become the standard for cross-modal image-text representation learning. Improving CLIP typically requires additional data and ret…
Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretraining
Bingqian Lin, Zicong Chen, Mingjie Li +13
Medical artificial general intelligence (MAGI) enables one foundation model to solve different medical tasks, which is very practical in the medical domain. It can significantly re…
DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignment
Lewei Yao, Jianhua Han, Xiaodan Liang +4
This paper presents DetCLIPv2, an efficient and scalable training framework that incorporates large-scale image-text pairs to achieve open-vocabulary object detection (OVD). Unlike…
LiDAR-NeRF: Novel LiDAR View Synthesis via Neural Radiance Fields
Tang Tao, Longfei Gao, Guangrun Wang +7
We introduce a new task, novel view synthesis for LiDAR sensors. While traditional model-based LiDAR simulators with style-transfer neural networks can be applied to render novel v…
CLIP: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data
Yihan Zeng, Chenhan Jiang, Jiageng Mao +7
Contrastive Language-Image Pre-training, benefiting from large-scale unlabeled text-image pairs, has demonstrated great performance in open-world vision understanding tasks. Howeve…
GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning
Zhenyu Xie, Zaiyu Huang, Xin Dong +5
Image-based Virtual Try-ON aims to transfer an in-shop garment onto a specific person. Existing methods employ a global warping module to model the anisotropic deformation for diff…
CapDet: Unifying Dense Captioning and Open-World Detection Pretraining
Yanxin Long, Youpeng Wen, Jianhua Han +5
Benefiting from large-scale vision-language pre-training on image-text pairs, open-world detection methods have shown superior generalization ability under the zero-shot or few-sho…
ViewCo: Discovering Text-Supervised Segmentation Masks via Multi-View Semantic Consistency
Pengzhen Ren, Changlin Li, Hang Xu +5
Recently, great success has been made in learning visual representations from text supervision, facilitating the emergence of text-supervised semantic segmentation. However, existi…