16 papers
MORE: Learning Motion-Oriented Registers for Driving World Models and Planning
Yingdong Gu, Haoyu Wang, Shaocheng Yan +9
By predicting future states, latent world models provide rich self-supervision for end-to-end driving, but often inherit dense features from pretrained visual encoders as their wor…
Commanding at the Speed of Sound: Audio-Centric Referring Expression Segmentation for Aerial Scenes
Fan Liu, Ziyun Chen, Yijun Wang +6
Referring expression segmentation (RES) aims to localize targets at the pixel level guided by human instructions. However, current models predominantly follow a text-centric paradi…
DroneWAM: Efficient World Action Model for Drone Visual Navigation
Liang Yao, Fan Liu, Hongbo Lu +5
World-action models give visual navigation agents a way to anticipate how candidate actions will change future observations and to act from the predicted consequences. For drones,…
RISE: Adaptive Imagination for World Action Models
Hongbo Lu, Liang Yao, Chenghao He +5
World Action Models (WAMs) improve planning by incorporating future world evolution into action generation, yet existing methods allocate a fixed imagination budget to every scene.…
The DAWN of World-Action Interactive Models
Hongbo Lu, Liang Yao, Chenghao He +6
A plausible scene evolution depends on the maneuver being considered, while a good maneuver depends on how the scene may evolve. Existing World Action Models (WAMs) largely miss th…
VisionNVS: Self-Supervised Inpainting for Novel View Synthesis under the Virtual-Shift Paradigm
Hongbo Lu, Liang Yao, Chenghao He +4
A fundamental bottleneck in Novel View Synthesis (NVS) for autonomous driving is the inherent supervision gap on novel trajectories: models are tasked with synthesizing unseen view…
Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs
Huan Zheng, Yucheng Zhou, Tianyi Yan +6
Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in medical image analysis. However, their application in gastrointestinal endoscopy is currently hin…
WildSVG: Towards Reliable SVG Generation Under Real-Word Conditions
Marco Terral, Haotian Zhang, Tianyang Zhang +8
We introduce the task of SVG extraction, which consists in translating specific visual inputs from an image into scalable vector graphics. Existing multimodal models achieve strong…
Semantic Causality-Aware Vision-Based 3D Occupancy Prediction
Dubing Chen, Huan Zheng, Yucheng Zhou +5
Vision-based 3D semantic occupancy prediction is a critical task in 3D vision that integrates volumetric 3D reconstruction with semantic understanding. Existing methods, however, o…
Two Causes, Not One: Rethinking Omission and Fabrication Hallucinations in MLLMs
Guangzong Si, Hao Yin, Xianfei Li +4
Multimodal Large Language Models (MLLMs) have achieved impressive advances, yet object hallucination remains a persistent challenge. Existing methods, based on the flawed assumptio…
Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Prediction
Dubing Chen, Huan Zheng, Jin Fang +6
We present GDFusion, a temporal fusion method for vision-based 3D semantic occupancy prediction (VisionOcc). GDFusion opens up the underexplored aspects of temporal fusion within t…
You Only Click Once: Single Point Weakly Supervised 3D Instance Segmentation for Autonomous Driving
Guangfeng Jiang, Jun Liu, Yongxuan Lv +5
Outdoor LiDAR point cloud 3D instance segmentation is a crucial task in autonomous driving. However, it requires laborious human efforts to annotate the point cloud for training a…
Int2Planner: An Intention-based Multi-modal Motion Planner for Integrated Prediction and Planning
Xiaolei Chen, Junchi Yan, Wenlong Liao +2
Motion planning is a critical module in autonomous driving, with the primary challenge of uncertainty caused by interactions with other participants. As most previous methods treat…
Generative Planning with 3D-vision Language Pre-training for End-to-End Autonomous Driving
Tengpeng Li, Hanli Wang, Xianfei Li +3
Autonomous driving is a challenging task that requires perceiving and understanding the surrounding environment for safe trajectory planning. While existing vision-based end-to-end…
Open-Vocabulary Object Detection via Neighboring Region Attention Alignment
Sunyuan Qiang, Xianfei Li, Yanyan Liang +3
The nature of diversity in real-world environments necessitates neural network models to expand from closed category settings to accommodate novel emerging categories. In this pape…
MWSIS: Multimodal Weakly Supervised Instance Segmentation with 2D Box Annotations for Autonomous Driving
Guangfeng Jiang, Jun Liu, Yuzhi Wu +3
Instance segmentation is a fundamental research in computer vision, especially in autonomous driving. However, manual mask annotation for instance segmentation is quite time-consum…