YOLOP: You Only Look Once for Panoptic Driving Perception

Abstract

A panoptic driving perception system is an essential part of autonomous driving. A high-precision and real-time perception system can assist the vehicle in making the reasonable decision while driving. We present a panoptic driving perception network (YOLOP) to perform traffic object detection, drivable area segmentation and lane detection simultaneously. It is composed of one encoder for feature extraction and three decoders to handle the specific tasks. Our model performs extremely well on the challenging BDD100K dataset, achieving state-of-the-art on all three tasks in terms of accuracy and speed. Besides, we verify the effectiveness of our multi-task learning model for joint training via ablative studies. To our best knowledge, this is the first work that can process these three visual perception tasks simultaneously in real-time on an embedded device Jetson TX2(23 FPS) and maintain excellent accuracy. To facilitate further research, the source codes and pre-trained models are released at https://github.com/hustvl/YOLOP.

References in corpus (7)

• SSD: Single Shot MultiBox Detector
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• YOLOv4: Optimal Speed and Accuracy of Object Detection
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning
• Location-Sensitive Visual Recognition with Cross-IOU Loss

Cited by in corpus (12)

• TriLiteNet: Lightweight Model for Multi-Task Visual Perception
• Implementation of a perception system for autonomous vehicles using a detection-segmentation network in SoC FPGA
• Efficient Perception, Planning, and Control Algorithm for Vision-Based Automated Vehicles
• Prompt-Driven Building Footprint Extraction in Aerial Images with Offset-Building Model
• Detection-segmentation convolutional neural network for autonomous vehicle perception
• Improving Road Segmentation in Challenging Domains Using Similar Place Priors
• Pedestrian Emergency Braking in Ten Weeks
• Multimodal-Enhanced Objectness Learner for Corner Case Detection in Autonomous Driving
• Minimal Sufficient Views: A DNN model making predictions with more evidence has higher accuracy
• FiGKD: Fine-Grained Knowledge Distillation via High-Frequency Detail Transfer
• TwinMixing: A Shuffle-Aware Feature Interaction Model for Multi-Task Segmentation
• Edge Deployable Lightweight Framework for Real-Time Driver Safety System with Integrated Lane Departure and Traffic Sign Recognition