EEG-Driven Decoding Framework for Passenger Hazard Perception in Highly Automated Vehicles
arXiv:2609.07128
Abstract
Reliable risk assessment remains a central challenge for Autonomous Vehicles (AVs). Despite advances in automation, passenger cognition provides a non-intrusive auxiliary signal that improves both objective and perceived safety without requiring active human intervention. We introduce an Electroencephalogram (EEG)-based Brain-Computer Interface (BCI) that decodes passenger neural responses for both Risk Prediction (RP) and Danger Identification (DI), explicitly modeling humans as passengers to match real-world AV use. To achieve this, we propose the Passenger Cognitive Model (PCM), Risk-aware Sequential Labeling (RSL), and the Passenger EEG Decoding Strategy (PEDS), which integrates a 3D Convolutional Recurrent Neural Network (3D-CRNN) model for joint EEG decoding. Experimental results show that 3D-CRNN achieves a Balanced Accuracy (BA) of in RP and improves single-subject DI from to with RSL. Event-wise analyses further show that 3D-CRNN consistently outperforms other models across different event types in RP and DI. In generalization experiments, 3D-CRNN achieves BA in cross-session DI and BA on seen subjects in cross-subject evaluation, while maintaining a BA on unseen subjects, demonstrating promising generalizability and transferability across both intra-subject and inter-subject variability. These findings establish an Electroencephalogram (EEG) decoding framework for AV passenger hazard perception and suggest that passenger cognitive signals can provide auxiliary supervision for future AV decision-making and Safety of the Intended Functionality (SOTIF) support.
31 pages, 8 figures, 13 tables, including appendices. Accepted for publication in Automotive Innovation. Yingkai Yang and Ashton Yu Xuan Tan contributed equally. Corresponding author: Hong Wang. Data: https://doi.org/10.21227/jw72-m261 ; Code: https://github.com/SOTIF-AVLab/EEG2023