Towards Understanding the Benefits of Neural Network Parameterizations in Geophysical Inversions: A Study With Neural Fields

Abstract

In this work, we employ neural fields, which use neural networks to map a coordinate to the corresponding physical property value at that coordinate, in a test-time learning manner. For a test-time learning method, the weights are learned during the inversion, as compared to traditional approaches which require a network to be trained using a training dataset. Results for synthetic examples in seismic tomography and direct current resistivity inversions are shown first. We then perform a singular value decomposition analysis on the Jacobian of the weights of the neural network (SVD analysis) for both cases to explore the effects of neural networks on the recovered model. The results show that the test-time learning approach can eliminate unwanted artifacts in the recovered subsurface physical property model caused by the sensitivity of the survey and physics. Therefore, NFs-Inv improves the inversion results compared to the conventional inversion in some cases such as the recovery of the dip angle or the prediction of the boundaries of the main target. In the SVD analysis, we observe similar patterns in the left-singular vectors as were observed in some diffusion models, trained in a supervised manner, for generative tasks in computer vision. This observation provides evidence that there is an implicit bias, which is inherent in neural network structures, that is useful in supervised learning and test-time learning models. This implicit bias has the potential to be useful for recovering models in geophysical inversions.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Scaling Laws for Neural Language Models
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• A Convergence Theory for Deep Learning via Over-Parameterization
• Physics-informed Neural Networks (PINNs) for Wave Propagation and Full Waveform Inversions
• Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network
• Implicit Full Waveform Inversion with Deep Neural Representation
• Rethinking Positional Encoding
• Integrating Deep Neural Networks with Full-waveform Inversion: Reparametrization, Regularization, and Uncertainty Quantification
• Automatic Differentiation-based Full Waveform Inversion with Flexible Workflows
• Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data
• A Test-Time Learning Approach to Reparameterize the Geophysical Inverse Problem with a Convolutional Neural Network