Group-level Brain Decoding with Deep Learning

Abstract

Decoding brain imaging data are gaining popularity, with applications in brain-computer interfaces and the study of neural representations. Decoding is typicallysubject-specific and does not generalise well over subjects, due to high amounts ofbetween subject variability. Techniques that overcome this will not only providericher neuroscientific insights but also make it possible for group-level models to out-perform subject-specific models. Here, we propose a method that uses subjectembedding, analogous to word embedding in natural language processing, to learnand exploit the structure in between-subject variability as part of a decoding model,our adaptation of the WaveNet architecture for classification. We apply this to mag-netoencephalography data, where 15 subjects viewed 118 different images, with30 examples per image; to classify images using the entire 1 s window followingimage presentation. We show that the combination of deep learning and subjectembedding is crucial to closing the performance gap between subject- and group-level decoding models. Importantly, group models outperform subject models onlow-accuracy subjects (although slightly impair high-accuracy subjects) and can behelpful for initialising subject models. While we have not generally found group-levelmodels to perform better than subject-level models, the performance of groupmodelling is expected to be even higher with bigger datasets. In order to providephysiological interpretation at the group level, we make use of permutation featureimportance. This provides insights into the spatiotemporal and spectral informationencoded in the models. All code is available on GitHub (https://github.com/ricsinaruto/MEG-group-decode).

arXiv commentPublished in Human Brain Mapping

References in corpus (10)

• Scikit-learn: Machine Learning in Python
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Array Programming with NumPy
• EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces
• Deep learning with convolutional neural networks for EEG decoding and visualization
• Interpretable machine learning: definitions, methods, and applications
• Robust Speech Recognition via Large-Scale Weak Supervision
• Decoding speech perception from non-invasive brain recordings
• Mind Reader: Reconstructing complex images from brain activities
• Group-level Brain Decoding with Deep Learning

Cited by in corpus (1)

• Group-level Brain Decoding with Deep Learning