Predicting Different Acoustic Features from EEG and towards direct synthesis of Audio Waveform from EEG
arXiv:2006.01262
Abstract
In [1,2] authors provided preliminary results for synthesizing speech from electroencephalography (EEG) features where they first predict acoustic features from EEG features and then the speech is reconstructed from the predicted acoustic features using griffin lim reconstruction algorithm. In this paper we first introduce a deep learning model that takes raw EEG waveform signals as input and directly produces audio waveform as output. We then demonstrate predicting 16 different acoustic features from EEG features. We demonstrate our results for both spoken and listen condition in this paper. The results presented in this paper shows how different acoustic features are related to non-invasive neural EEG signals recorded during speech perception and production.
arXiv commentUnder Review
References in corpus (5)
- Adam: A Method for Stochastic Optimization
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
- State-of-the-art Speech Recognition using EEG and Towards Decoding of Speech Spectrum From EEG
- Improving EEG based Continuous Speech Recognition