Where and What: Driver Attention-based Object Detection

Abstract

Human drivers use their attentional mechanisms to focus on critical objects and make decisions while driving. As human attention can be revealed from gaze data, capturing and analyzing gaze information has emerged in recent years to benefit autonomous driving technology. Previous works in this context have primarily aimed at predicting "where" human drivers look at and lack knowledge of "what" objects drivers focus on. Our work bridges the gap between pixel-level and object-level attention prediction. Specifically, we propose to integrate an attention prediction module into a pretrained object detection framework and predict the attention in a grid-based style. Furthermore, critical objects are recognized based on predicted attended-to areas. We evaluate our proposed method on two driver attention datasets, BDD-A and DR(eye)VE. Our framework achieves competitive state-of-the-art performance in the attention prediction on both pixel-level and object-level but is far more efficient (75.3 GFLOPs less) in computation.

22 pages

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• YOLOv4: Optimal Speed and Accuracy of Object Detection
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• DeepGaze II: Reading fixations from deep features trained on object recognition
• Human Attention in Fine-grained Classification
• See, Attend and Brake: An Attention-based Saliency Map Prediction Model for End-to-End Driving