Structure-Preserving Transformers for Sequences of SPD Matrices

Abstract

In recent years, Transformer-based auto-attention mechanisms have been successfully applied to the analysis of a variety of context-reliant data types, from texts to images and beyond, including data from non-Euclidean geometries. In this paper, we present such a mechanism, designed to classify sequences of Symmetric Positive Definite matrices while preserving their Riemannian geometry throughout the analysis. We apply our method to automatic sleep staging on timeseries of EEG-derived covariance matrices from a standard dataset, obtaining high levels of stage-wise performance.

New year, new version! (updated template, minimal additions - including two new references)

References in corpus (7)

• DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG
• SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification
• XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging
• Intra- and Inter-epoch Temporal Context Network (IITNet) Using Sub-epoch Features for Automatic Sleep Scoring on Raw Single-channel EEG
• Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions
• MAtt: A Manifold Attention Network for EEG Decoding
• L-SeqSleepNet: Whole-cycle Long Sequence Modelling for Automatic Sleep Staging