SalientSleepNet: Multimodal Salient Wave Detection Network for Sleep Staging

Abstract

Sleep staging is fundamental for sleep assessment and disease diagnosis. Although previous attempts to classify sleep stages have achieved high classification performance, several challenges remain open: 1) How to effectively extract salient waves in multimodal sleep data; 2) How to capture the multi-scale transition rules among sleep stages; 3) How to adaptively seize the key role of specific modality for sleep staging. To address these challenges, we propose SalientSleepNet, a multimodal salient wave detection network for sleep staging. Specifically, SalientSleepNet is a temporal fully convolutional network based on the ${\mathrm{U}}^2$-Net architecture that is originally proposed for salient object detection in computer vision. It is mainly composed of two independent ${\mathrm{U}}^2$-like streams to extract the salient features from multimodal data, respectively. Meanwhile, the multi-scale extraction module is designed to capture multi-scale transition rules among sleep stages. Besides, the multimodal attention module is proposed to adaptively capture valuable information from multimodal data for the specific sleep stage. Experiments on the two datasets demonstrate that SalientSleepNet outperforms the state-of-the-art baselines. It is worth noting that this model has the least amount of parameters compared with the existing deep neural network models.

arXiv commentAccepted by IJCAI 2021. The SOLE copyright holder is IJCAI (International Joint Conferences on Artificial Intelligence), all rights reserved

References in corpus (8)

• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• U${}^2$-Net: Going Deeper with Nested U-Structure for Salient Object Detection
• DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG
• SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging
• A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series
• SleepEEGNet: Automated Sleep Stage Scoring with Sequence to Sequence Deep Learning Approach
• XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging
• U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging

Cited by in corpus (2)

• HetEmotionNet: Two-Stream Heterogeneous Graph Recurrent Neural Network for Multi-modal Emotion Recognition
• Multi-View Spatial-Temporal Graph Convolutional Networks with Domain Generalization for Sleep Stage Classification