BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning

Abstract

Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving. Researchers are usually constrained to study a small set of problems on one dataset, while real-world computer vision applications require performing tasks of various complexities. We construct BDD100K, the largest driving video dataset with 100K videos and 10 tasks to evaluate the exciting progress of image recognition algorithms on autonomous driving. The dataset possesses geographic, environmental, and weather diversity, which is useful for training models that are less likely to be surprised by new conditions. Based on this diverse dataset, we build a benchmark for heterogeneous multitask learning and study how to solve the tasks together. Our experiments show that special training strategies are needed for existing models to perform such heterogeneous tasks. BDD100K opens the door for future studies in this important venue.

arXiv commentPublished at IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020

References in corpus (9)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• MOT16: A Benchmark for Multi-Object Tracking
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• YouTube-8M: A Large-Scale Video Classification Benchmark
• Learning multiple visual domains with residual adapters
• The Natural Language Decathlon: Multitask Learning as Question Answering
• YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark
• Scalability in Perception for Autonomous Driving: Waymo Open Dataset

Cited by in corpus (8)

• YOLOP: You Only Look Once for Panoptic Driving Perception
• A Survey of End-to-End Driving: Architectures and Training Methods
• Small Object Detection: A Comprehensive Survey on Challenges, Techniques and Real-World Applications
• A Unified Longitudinal Trajectory Dataset for Automated Vehicle
• Unsupervised Traffic Scene Generation with Synthetic 3D Scene Graphs
• On the Road with 16 Neurons: Mental Imagery with Bio-inspired Deep Neural Networks
• Workshop on Autonomous Driving at CVPR 2021: Technical Report for Streaming Perception Challenge
• Detecting Out-of-distribution Objects Using Neuron Activation Patterns