SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification

Abstract

Background: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. Methods: Towards interpretability, this work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the model's decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the model's decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. Results: Making sense of the transformer's self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. Conclusion: Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes. Significance: Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.

This article has been published in IEEE Transactions on Biomedical Engineering

References in corpus (9)

• A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series
• SleepEEGNet: Automated Sleep Stage Scoring with Sequence to Sequence Deep Learning Approach
• Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks
• Automatic sleep stage classification with deep residual networks in a mixed-cohort setting
• U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging
• Personalized Automatic Sleep Staging with Single-Night Data: a Pilot Study with KL-Divergence Regularization
• Personalizing deep learning models for automatic sleep staging
• TRIER: Template-Guided Neural Networks for Robust and Interpretable Sleep Stage Identification from EEG Recordings
• Light-weight sleep monitoring: electrode distance matters more than placement for automatic scoring

Cited by in corpus (19)

• Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions
• SleePyCo: Automatic Sleep Scoring with Feature Pyramid and Contrastive Learning
• Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers
• Do Not Sleep on Traditional Machine Learning: Simple and Interpretable Techniques Are Competitive to Deep Learning for Sleep Scoring
• PrecTime: A Deep Learning Architecture for Precise Time Series Segmentation in Industrial Manufacturing Operations
• Pediatric Automatic Sleep Staging: A comparative study of state-of-the-art deep learning methods
• Multi-Task Learning for Arousal and Sleep Stage Detection Using Fully Convolutional Networks
• Fast Entropy-Based Methods of Word-Level Confidence Estimation for End-To-End Automatic Speech Recognition
• Improving Transformer Performance for French Clinical Notes Classification Using Mixture of Experts on a Limited Dataset
• aSAGA: Automatic Sleep Analysis with Gray Areas
• SERF: Interpretable Sleep Staging using Embeddings, Rules, and Features
• A Deep Generative Model for Five-Class Sleep Staging with Arbitrary Sensor Input
• CSLP-AE: A Contrastive Split-Latent Permutation Autoencoder Framework for Zero-Shot Electroencephalography Signal Conversion
• Mamba-based Deep Learning Approach for Sleep Staging on a Wireless Multimodal Wearable System without Electroencephalography
• sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging
• NeuroSleepNet: An Explainable Multi-Head Attention-Based Framework With Spatial and Multi-Scale Independent Temporal Context Learning for Automatic Sleep Stage Scoring
• Contrastive Learning for Sleep Staging based on Inter Subject Correlation
• Atoms of Thought: Universal EEG Representation Learning with Microstates
• Structure-Preserving Transformers for Sequences of SPD Matrices