Benchmark for Models Predicting Human Behavior in Gap Acceptance Scenarios

Abstract

Autonomous vehicles currently suffer from a time-inefficient driving style caused by uncertainty about human behavior in traffic interactions. Accurate and reliable prediction models enabling more efficient trajectory planning could make autonomous vehicles more assertive in such interactions. However, the evaluation of such models is commonly oversimplistic, ignoring the asymmetric importance of prediction errors and the heterogeneity of the datasets used for testing. We examine the potential of recasting interactions between vehicles as gap acceptance scenarios and evaluating models in this structured environment. To that end, we develop a framework aiming to facilitate the evaluation of any model, by any metric, and in any scenario. We then apply this framework to state-of-the-art prediction models, which all show themselves to be unreliable in the most safety-critical situations.

arXiv comment11 pages, 5 figures, accepted by in IEEE Transactions on Intelligent Vehicles, 2023

References in corpus (11)

• Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review
• Human Motion Trajectory Prediction: A Survey
• The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
• Pedestrian Models for Autonomous Driving Part II: High-Level Models of Human Behavior
• VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change
• LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving
• Early Lane Change Prediction for Automated Driving Systems Using Multi-Task Attention-based Convolutional Neural Networks
• Predicting highway lane-changing maneuvers: A benchmark analysis of machine and ensemble learning algorithms
• Beyond RMSE: Do machine-learned models of road user interaction produce human-like behavior?
• Comparing merging behaviors observed in naturalistic data with behaviors generated by a machine learned model
• Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles