56 citations · 262 across the 66 of their papers we have counts for
43 papers · 1 filter
RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Grasping
Dongming Wu, Yanping Fu, Saike Huang +8
General robotic grasping systems require accurate object affordance perception in diverse open-world scenarios following human instructions. However, current studies suffer from th…
Hita: Holistic Tokenizer for Autoregressive Image Generation
Anlin Zheng, Haochen Wang, Yucheng Zhao +4
Vanilla autoregressive image generation models generate visual tokens step-by-step, limiting their ability to capture holistic relationships among token sequences. Moreover, becaus…
Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generation
Anlin Zheng, Xin Wen, Xuanyang Zhang +5
In this work, we present a novel direction to build an image tokenizer directly on top of a frozen vision foundation model, which is a largely underexplored area. Specifically, we…
DEGround: An Effective Baseline for Ego-centric 3D Visual Grounding with a Homogeneous Framework
Yani Zhang, Dongming Wu, Hao Shi +3
A core task in embodied intelligence is ego-centric 3D visual grounding. Existing methods typically adopt two-stage, heterogeneous pipelines that pair a detector with a separate gr…
PADriver: Towards Personalized Autonomous Driving
Genghua Kou, Fan Jia, Weixin Mao +7
In this paper, we propose PADriver, a novel closed-loop framework for personalized autonomous driving (PAD). Built upon Multi-modal Large Language Model (MLLM), PADriver takes stre…
Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness
Haochen Wang, Yucheng Zhao, Tiancai Wang +3
The rapid development of Large Multimodal Models (LMMs) for 2D images and videos has spurred efforts to adapt these models for interpreting 3D scenes. However, the absence of large…
UniScene: Unified Occupancy-centric Driving Scene Generation
Bohan Li, Jiazhe Guo, Hongsi Liu +14
Generating high-fidelity, controllable, and annotated training data is critical for autonomous driving. Existing methods typically generate a single data form directly from a coars…
Continual LLaVA: Continual Instruction Tuning in Large Vision-Language Models
Meng Cao, Yuyang Liu, Yingfei Liu +6
Instruction tuning constitutes a prevalent technique for tailoring Large Vision Language Models (LVLMs) to meet individual task requirements. To date, most of the existing approach…
Reconstructive Visual Instruction Tuning
Haochen Wang, Anlin Zheng, Yucheng Zhao +4
This paper introduces reconstructive visual instruction tuning (ROSS), a family of Large Multimodal Models (LMMs) that exploit vision-centric supervision signals. In contrast to co…
Panacea+: Panoramic and Controllable Video Generation for Autonomous Driving
Yuqing Wen, Yucheng Zhao, Yingfei Liu +7
The field of autonomous driving increasingly demands high-quality annotated video training data. In this paper, we propose Panacea+, a powerful and universally applicable framework…
The RoboDrive Challenge: Drive Anytime Anywhere in Any Condition
Lingdong Kong, Shaoyuan Xie, Hanjiang Hu +88
In the realm of autonomous driving, robust perception under out-of-distribution conditions is paramount for the safe deployment of vehicles. Challenges such as adverse weather, sen…
Is a 3D-Tokenized LLM the Key to Reliable Autonomous Driving?
Yifan Bai, Dongming Wu, Yingfei Liu +8
Rapid advancements in Autonomous Driving (AD) tasks turned a significant shift toward end-to-end fashion, particularly in the utilization of vision-language models (VLMs) that inte…
SubjectDrive: Scaling Generative Data in Autonomous Driving via Subject Control
Binyuan Huang, Yuqing Wen, Yucheng Zhao +9
Autonomous driving progress relies on large-scale annotated datasets. In this work, we explore the potential of generative models to produce vast quantities of freely-labeled data…
Stream Query Denoising for Vectorized HD Map Construction
Shuo Wang, Fan Jia, Yingfei Liu +6
To enhance perception performance in complex and extensive scenarios within the realm of autonomous driving, there has been a noteworthy focus on temporal modeling, with a particul…
Bootstrap Masked Visual Modeling via Hard Patches Mining
Haochen Wang, Junsong Fan, Yuxi Wang +4
Masked visual modeling has attracted much attention due to its promising potential in learning generalizable representations. Typical approaches urge models to predict specific con…
Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future
Hongyang Li, Yang Li, Huijie Wang +16
With the continuous maturation and application of autonomous driving technology, a systematic examination of open-source autonomous driving datasets becomes instrumental in fosteri…
Merlin:Empowering Multimodal LLMs with Foresight Minds
En Yu, Liang Zhao, Yana Wei +8
Humans possess the remarkable ability to foresee the future to a certain extent based on present observations, a skill we term as foresight minds. However, this capability remains…
Panacea: Panoramic and Controllable Video Generation for Autonomous Driving
Yuqing Wen, Yucheng Zhao, Yingfei Liu +7
The field of autonomous driving increasingly demands high-quality annotated training data. In this paper, we propose Panacea, an innovative approach to generate panoramic and contr…
ADriver-I: A General World Model for Autonomous Driving
Fan Jia, Weixin Mao, Yingfei Liu +5
Typically, autonomous driving adopts a modular design, which divides the full stack into perception, prediction, planning and control parts. Though interpretable, such modular desi…
VLM-Eval: A General Evaluation on Video Large Language Models
Shuailin Li, Yuang Zhang, Yucheng Zhao +4
Despite the rapid development of video Large Language Models (LLMs), a comprehensive evaluation is still absent. In this paper, we introduce a unified evaluation that encompasses m…
TopoMLP: A Simple yet Strong Pipeline for Driving Topology Reasoning
Dongming Wu, Jiahao Chang, Fan Jia +3
Topology reasoning aims to comprehensively understand road scenes and present drivable routes in autonomous driving. It requires detecting road centerlines (lane) and traffic eleme…
Language Prompt for Autonomous Driving
Dongming Wu, Wencheng Han, Yingfei Liu +4
A new trend in the computer vision community is to capture objects of interest following flexible human command represented by a natural language prompt. However, the progress of u…
Far3D: Expanding the Horizon for Surround-view 3D Object Detection
Xiaohui Jiang, Shuailin Li, Yingfei Liu +5
Recently 3D object detection from surround-view images has made notable advancements with its low deployment cost. However, most works have primarily focused on close perception ra…
OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentation
Dongming Wu, Tiancai Wang, Yuang Zhang +2
Referring video object segmentation (RVOS) aims at segmenting an object in a video following human instruction. Current state-of-the-art methods fall into an offline pattern, in wh…