Complex Gated Recurrent Neural Networks

Abstract

Complex numbers have long been favoured for digital signal processing, yet complex representations rarely appear in deep learning architectures. RNNs, widely used to process time series and sequence information, could greatly benefit from complex representations. We present a novel complex gated recurrent cell, which is a hybrid cell combining complex-valued and norm-preserving state transitions with a gating mechanism. The resulting RNN exhibits excellent stability and convergence properties and performs competitively on the synthetic memory and adding task, as well as on the real-world tasks of human motion prediction.

Cited by in corpus (10)

• Towards Understanding Theoretical Advantages of Complex-Reaction Networks
• 3D-Rotation-Equivariant Quaternion Neural Networks
• Multiresolution Transformer Networks: Recurrence is Not Essential for Modeling Hierarchical Structure
• Complex Unitary Recurrent Neural Networks using Scaled Cayley Transform
• Complex Transformer: A Framework for Modeling Complex-Valued Sequence
• Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory
• Neural Networks with Complex-Valued Weights Have No Spurious Local Minima
• FallDeF5: A Fall Detection Framework Using 5G-based Deep Gated Recurrent Unit Networks
• Sequence Prediction using Spectral RNNs
• Bayesian Sparsification Methods for Deep Complex-valued Networks