XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging

Abstract

Automating sleep staging is vital to scale up sleep assessment and diagnosis to serve millions experiencing sleep deprivation and disorders and enable longitudinal sleep monitoring in home environments. Learning from raw polysomnography signals and their derived time-frequency image representations has been prevalent. However, learning from multi-view inputs (e.g., both the raw signals and the time-frequency images) for sleep staging is difficult and not well understood. This work proposes a sequence-to-sequence sleep staging model, XSleepNet, that is capable of learning a joint representation from both raw signals and time-frequency images. Since different views may generalize or overfit at different rates, the proposed network is trained such that the learning pace on each view is adapted based on their generalization/overfitting behavior. In simple terms, the learning on a particular view is speeded up when it is generalizing well and slowed down when it is overfitting. View-specific generalization/overfitting measures are computed on-the-fly during the training course and used to derive weights to blend the gradients from different views. As a result, the network is able to retain the representation power of different views in the joint features which represent the underlying distribution better than those learned by each individual view alone. Furthermore, the XSleepNet architecture is principally designed to gain robustness to the amount of training data and to increase the complementarity between the input views. Experimental results on five databases of different sizes show that XSleepNet consistently outperforms the single-view baselines and the multi-view baseline with a simple fusion strategy. Finally, XSleepNet also outperforms prior sleep staging methods and improves previous state-of-the-art results on the experimental databases.

This article has been published in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Fully Convolutional Networks for Semantic Segmentation
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG
• SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging
• A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series
• Contrastive Multiview Coding
• SleepEEGNet: Automated Sleep Stage Scoring with Sequence to Sequence Deep Learning Approach
• Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification
• Neural network an1alysis of sleep stages enables efficient diagnosis of narcolepsy
• Mixed Neural Network Approach for Temporal Sleep Stage Classification
• Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks
• Intra- and Inter-epoch Temporal Context Network (IITNet) Using Sub-epoch Features for Automatic Sleep Scoring on Raw Single-channel EEG
• Towards More Accurate Automatic Sleep Staging via Deep Transfer Learning
• SLEEPNET: Automated Sleep Staging System via Deep Learning
• U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging
• Personalized Automatic Sleep Staging with Single-Night Data: a Pilot Study with KL-Divergence Regularization
• Personalizing deep learning models for automatic sleep staging
• Towards a Flexible Deep Learning Method for Automatic Detection of Clinically Relevant Multi-Modal Events in the Polysomnogram

Cited by in corpus (24)

• SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification
• MIN2Net: End-to-End Multi-Task Learning for Subject-Independent Motor Imagery EEG Classification
• Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions
• Change Point Detection in Time Series Data using Autoencoders with a Time-Invariant Representation
• SleePyCo: Automatic Sleep Scoring with Feature Pyramid and Contrastive Learning
• Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers
• Do Not Sleep on Traditional Machine Learning: Simple and Interpretable Techniques Are Competitive to Deep Learning for Sleep Scoring
• U-Sleep's resilience to AASM guidelines
• Pediatric Automatic Sleep Staging: A comparative study of state-of-the-art deep learning methods
• Multi-Scored Sleep Databases: How to Exploit the Multiple-Labels in Automated Sleep Scoring
• MixNet: Joining Force of Classical and Modern Approaches Toward the Comprehensive Pipeline in Motor Imagery EEG Classification
• Deep Efficient Continuous Manifold Learning for Time Series Modeling
• Ubi-SleepNet: Advanced Multimodal Fusion Techniques for Three-stage Sleep Classification Using Ubiquitous Sensing
• Multi-Task Learning for Arousal and Sleep Stage Detection Using Fully Convolutional Networks
• SalientSleepNet: Multimodal Salient Wave Detection Network for Sleep Staging
• Data augmentation for learning predictive models on EEG: a systematic comparison
• A Deep Generative Model for Five-Class Sleep Staging with Arbitrary Sensor Input
• Evaluating the structure of cognitive tasks with transfer learning
• CSLP-AE: A Contrastive Split-Latent Permutation Autoencoder Framework for Zero-Shot Electroencephalography Signal Conversion
• AI Generalisation Gap In Comorbid Sleep Disorder Staging
• sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging
• DeepSleepNet-Lite: A Simplified Automatic Sleep Stage Scoring Model with Uncertainty Estimates
• NeuroSleepNet: An Explainable Multi-Head Attention-Based Framework With Spatial and Multi-Scale Independent Temporal Context Learning for Automatic Sleep Stage Scoring
• Structure-Preserving Transformers for Sequences of SPD Matrices