Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling

Abstract

Nonlocal neural networks have been proposed and shown to be effective in several computer vision tasks, where the nonlocal operations can directly capture long-range dependencies in the feature space. In this paper, we study the nature of diffusion and damping effect of nonlocal networks by doing spectrum analysis on the weight matrices of the well-trained networks, and then propose a new formulation of the nonlocal block. The new block not only learns the nonlocal interactions but also has stable dynamics, thus allowing deeper nonlocal structures. Moreover, we interpret our formulation from the general nonlocal modeling perspective, where we make connections between the proposed nonlocal network and other nonlocal models, such as nonlocal diffusion process and Markov jump process.

arXiv commentAccepted by NeurIPS 2018

Cited by in corpus (8)

• A Review on Deep Learning in Medical Image Reconstruction
• Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
• Is Attention Better Than Matrix Decomposition?
• Knowing What, Where and When to Look: Efficient Video Action Modeling with Attention
• Compact Global Descriptor for Neural Networks
• Interpolation between Residual and Non-Residual Networks
• A Spectral Nonlocal Block for Neural Networks
• Pseudoinverse Graph Convolutional Networks: Fast Filters Tailored for Large Eigengaps of Dense Graphs and Hypergraphs