Benchmark for Models Predicting Human Behavior in Gap Acceptance Scenarios

Abstract

Autonomous vehicles currently suffer from a time-inefficient driving style caused by uncertainty about human behavior in traffic interactions. Accurate and reliable prediction models enabling more efficient trajectory planning could make autonomous vehicles more assertive in such interactions. However, the evaluation of such models is commonly oversimplistic, ignoring the asymmetric importance of prediction errors and the heterogeneity of the datasets used for testing. We examine the potential of recasting interactions between vehicles as gap acceptance scenarios and evaluating models in this structured environment. To that end, we develop a framework aiming to facilitate the evaluation of any model, by any metric, and in any scenario. We then apply this framework to state-of-the-art prediction models, which all show themselves to be unreliable in the most safety-critical situations.

11 pages, 5 figures, accepted by in IEEE Transactions on Intelligent Vehicles, 2023

References in corpus (6)

• The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
• Early Lane Change Prediction for Automated Driving Systems Using Multi-Task Attention-based Convolutional Neural Networks
• Predicting highway lane-changing maneuvers: A benchmark analysis of machine and ensemble learning algorithms
• Beyond RMSE: Do machine-learned models of road user interaction produce human-like behavior?
• Comparing merging behaviors observed in naturalistic data with behaviors generated by a machine learned model
• Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles