104 citations · 147 across the 43 of their papers we have counts for
27 papers · 1 filter
FreeScale: Scaling 3D Scenes via Certainty-Aware Free-View Generation
Chenhan Jiang, Yu Chen, Qingwen Zhang +4
The development of generalizable Novel View Synthesis (NVS) models is critically limited by the scarcity of large-scale training data featuring diverse and precise camera trajector…
ECCV 2024 W-CODA: 1st Workshop on Multimodal Perception and Comprehension of Corner Cases in Autonomous Driving
Kai Chen, Ruiyuan Gao, Lanqing Hong +11
In this paper, we present details of the 1st W-CODA workshop, held in conjunction with the ECCV 2024. W-CODA aims to explore next-generation solutions for autonomous driving corner…
Learning 3D Persistent Embodied World Models
Siyuan Zhou, Yilun Du, Yuncong Yang +4
The ability to simulate the effects of future actions on the world is a crucial ability of intelligent embodied agents, enabling agents to anticipate the effects of their actions a…
CoherenDream: Boosting Holistic Text Coherence in 3D Generation via Multimodal Large Language Models Feedback
Chenhan Jiang, Yihan Zeng, Dit-Yan Yeung
Score Distillation Sampling (SDS) has achieved remarkable success in text-to-3D content generation. However, SDS-based methods struggle to maintain semantic fidelity for user promp…
Fourier Amplitude and Correlation Loss: Beyond Using L2 Loss for Skillful Precipitation Nowcasting
Chiu-Wai Yan, Shi Quan Foo, Van Hoan Trinh +3
Deep learning approaches have been widely adopted for precipitation nowcasting in recent years. Previous studies mainly focus on proposing new model architectures to improve pixel-…
Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models
Junjie Wu, Tsz Ting Chung, Kai Chen +1
Despite the outstanding performance in vision-language reasoning, Large Vision-Language Models (LVLMs) might generate hallucinated contents that do not exist in the given image. Mo…
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions
Kai Chen, Yunhao Gou, Runhui Huang +28
GPT-4o, an omni-modal model that enables vocal conversations with diverse emotions and tones, marks a milestone for omni-modal foundation models. However, empowering Large Language…
Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detection
Zhili Chen, Shuangjie Xu, Maosheng Ye +4
The Bird's-Eye-View (BEV) representation is a critical factor that directly impacts the 3D object detection performance, but the traditional BEV grid representation induces quadrat…
JointDreamer: Ensuring Geometry Consistency and Text Congruence in Text-to-3D Generation via Joint Score Distillation
Chenhan Jiang, Yihan Zeng, Tianyang Hu +4
Score Distillation Sampling (SDS) by well-trained 2D diffusion models has shown great promise in text-to-3D generation. However, this paradigm distills view-agnostic 2D image distr…
Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases
Kai Chen, Yanze Li, Wenhua Zhang +10
Large Vision-Language Models (LVLMs) have received widespread attention for advancing the interpretable self-driving. Existing evaluations of LVLMs primarily focus on multi-faceted…
DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception
Yibo Wang, Ruiyuan Gao, Kai Chen +8
Current perceptive models heavily depend on resource-intensive datasets, prompting the need for innovative solutions. Leveraging recent advances in diffusion models, synthetic data…
TransformMix: Learning Transformation and Mixing Strategies from Data
Tsz-Him Cheung, Dit-Yan Yeung
Data augmentation improves the generalization power of deep learning models by synthesizing more training samples. Sample-mixing is a popular data augmentation approach that create…
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation
Yunhao Gou, Kai Chen, Zhili Liu +6
Multimodal large language models (MLLMs) have shown impressive reasoning abilities. However, they are also more vulnerable to jailbreak attacks than their LLM predecessors. Althoug…
Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning
Yunhao Gou, Zhili Liu, Kai Chen +6
Instruction tuning of Large Vision-language Models (LVLMs) has revolutionized the development of versatile models with zero-shot generalization across a wide range of downstream vi…
TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models
Pengxiang Li, Kai Chen, Zhili Liu +7
Despite remarkable achievements in video synthesis, achieving granular control over complex dynamics, such as nuanced movement among multiple interacting objects, still presents a…
Gaussian Shell Maps for Efficient 3D Human Generation
Rameen Abdal, Wang Yifan, Zifan Shi +6
Efficient generation of 3D digital humans is important in several industries, including virtual reality, social media, and cinematic production. 3D generative adversarial networks…
Implicit Concept Removal of Diffusion Models
Zhili Liu, Kai Chen, Yifan Zhang +6
Text-to-image (T2I) diffusion models often inadvertently generate unwanted concepts such as watermarks and unsafe images. These concepts, termed as the "implicit concepts", could b…
MagicDrive: Street View Generation with Diverse 3D Geometry Control
Ruiyuan Gao, Kai Chen, Enze Xie +4
Recent advancements in diffusion models have significantly enhanced the data synthesis with 2D control. Yet, precise 3D control in street view generation, crucial for 3D perception…
SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation
Xuechao Chen, Shuangjie Xu, Xiaoyi Zou +3
LiDAR-based semantic perception tasks are critical yet challenging for autonomous driving. Due to the motion of objects and static/dynamic occlusion, temporal information plays an…
GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation
Kai Chen, Enze Xie, Zhe Chen +4
Diffusion models have attracted significant attention due to the remarkable ability to create content and generate data for tasks like image classification. However, the usage of d…
CLIP: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data
Yihan Zeng, Chenhan Jiang, Jiageng Mao +7
Contrastive Language-Image Pre-training, benefiting from large-scale unlabeled text-image pairs, has demonstrated great performance in open-world vision understanding tasks. Howeve…
Mixed Autoencoder for Self-supervised Visual Representation Learning
Kai Chen, Zhili Liu, Lanqing Hong +3
Masked Autoencoder (MAE) has demonstrated superior performance on various vision tasks via randomly masking image patches and reconstruction. However, effective data augmentation s…
Learning 3D-aware Image Synthesis with Unknown Pose Distribution
Zifan Shi, Yujun Shen, Yinghao Xu +5
Existing methods for 3D-aware image synthesis largely depend on the 3D pose distribution pre-estimated on the training set. An inaccurate estimation may mislead the model into lear…
Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator
Zifan Shi, Yinghao Xu, Yujun Shen +3
3D-aware image synthesis aims at learning a generative model that can render photo-realistic 2D images while capturing decent underlying 3D shapes. A popular solution is to adopt t…