Complex-valued neural networks for machine learning on non-stationary physical data

Abstract

Deep learning has become an area of interest in most scientific areas, including physical sciences. Modern networks apply real-valued transformations on the data. Particularly, convolutions in convolutional neural networks discard phase information entirely. Many deterministic signals, such as seismic data or electrical signals, contain significant information in the phase of the signal. We explore complex-valued deep convolutional networks to leverage non-linear feature maps. Seismic data commonly has a lowcut filter applied, to attenuate noise from ocean waves and similar long wavelength contributions. Discarding the phase information leads to low-frequency aliasing analogous to the Nyquist-Shannon theorem for high frequencies. In non-stationary data, the phase content can stabilize training and improve the generalizability of neural networks. While it has been shown that phase content can be restored in deep neural networks, we show how including phase information in feature maps improves both training and inference from deterministic physical data. Furthermore, we show that the reduction of parameters in a complex network outperforms larger real-valued networks.

arXiv comment17 pages total, 15 pages, 2 pages references, paper, 11 figures, 28 networks

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• WaveNet: A Generative Model for Raw Audio
• CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks
• SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• 70 years of machine learning in geoscience in review
• Seismic facies recognition based on prestack data using deep convolutional autoencoder
• Deep Complex Networks
• CT-image Super Resolution Using 3D Convolutional Neural Network
• Learning Representations Using Complex-Valued Nets

Cited by in corpus (4)

• Machine Learning in Nuclear Physics
• Machine Learning for the Detection and Identification of Internet of Things (IoT) Devices: A Survey
• The neural networks with tensor weights and emergent fermionic Wick rules in the large-width limit
• Adaptive Latent Space Tuning for Non-Stationary Distributions