Complex Transformer: A Framework for Modeling Complex-Valued Sequence
arXiv:1910.10202
Abstract
While deep learning has received a surge of interest in a variety of fields in recent years, major deep learning models barely use complex numbers. However, speech, signal and audio data are naturally complex-valued after Fourier Transform, and studies have shown a potentially richer representation of complex nets. In this paper, we propose a Complex Transformer, which incorporates the transformer model as a backbone for sequence modeling; we also develop attention and encoder-decoder network operating for complex input. The model achieves state-of-the-art performance on the MusicNet dataset and an In-phase Quadrature (IQ) signal dataset.
References in corpus (8)
- WaveNet: A Generative Model for Raw Audio
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
- XLNet: Generalized Autoregressive Pretraining for Language Understanding
- Deep Complex Networks
- Learning Features of Music from Scratch
- Complex Gated Recurrent Neural Networks
- Mathematical models for chemotaxis and their applications in self-organisation phenomena