A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspective
arXiv:2405.05173 · doi:10.1016/j.inffus.2024.102671
Abstract
3D occupancy perception technology aims to observe and understand dense 3D environments for autonomous vehicles. Owing to its comprehensive perception capability, this technology is emerging as a trend in autonomous driving perception systems, and is attracting significant attention from both industry and academia. Similar to traditional bird's-eye view (BEV) perception, 3D occupancy perception has the nature of multi-source input and the necessity for information fusion. However, the difference is that it captures vertical structures that are ignored by 2D BEV. In this survey, we review the most recent works on 3D occupancy perception, and provide in-depth analyses of methodologies with various input modalities. Specifically, we summarize general network pipelines, highlight information fusion techniques, and discuss effective network training. We evaluate and analyze the occupancy perception performance of the state-of-the-art on the most popular datasets. Furthermore, challenges and future research directions are discussed. We hope this paper will inspire the community and encourage more research work on 3D occupancy perception. A comprehensive list of studies in this survey is publicly available in an active repository that continuously collects the latest work: https://github.com/HuaiyuanXu/3D-Occupancy-Perception.
References in corpus (45)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Denoising Diffusion Probabilistic Models
- LLaMA: Open and Efficient Foundation Language Models
- Scaling Instruction-Finetuned Language Models
- Visual Instruction Tuning
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
- Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation
- MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
- BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View
- Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
- BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework
- Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
- BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving
- Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving
- Vision-Centric BEV Perception: A Survey
- OccDepth: A Depth-Aware Method for 3D Semantic Scene Completion
- Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformer
- FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation
- DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptation
- SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving
- PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Prediction
- POP-3D: Open-Vocabulary 3D Occupancy Prediction from Images
- OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments
- OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queries
- LiDAR-based 4D Occupancy Completion and Forecasting
- OVO: Open-Vocabulary Occupancy
- OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
- RoboBEV: Towards Robust Bird's Eye View Perception under Corruptions
- OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Prediction
- FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin
- Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perception
- SemFormer: Semantic Guided Activation Transformer for Weakly Supervised Semantic Segmentation
- Vision-based 3D occupancy prediction in autonomous driving: a review and outlook
- A Spatiotemporal Approach to Tri-Perspective Representation for 3D Semantic Occupancy Prediction
- Context and Geometry Aware Voxel Transformer for Semantic Scene Completion
- RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radar
- OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Driving
- OccFlowNet: Towards Self-supervised Occupancy Estimation via Differentiable Rendering and Occupancy Flow
- Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction
- InverseMatrixVT3D: An Efficient Projection Matrix-Based Approach for 3D Occupancy Prediction
- GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Prediction
- BDC-Occ: Binarized Deep Convolution Unit For Binarized Occupancy Network
- Occupancy as Set of Points
- Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Driving