PDE-based Group Equivariant Convolutional Neural Networks

Abstract

We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs). In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer's trainable weights. Formulating our PDEs on homogeneous spaces allows these networks to be designed with built-in symmetries such as rotation in addition to the standard translation equivariance of CNNs. Having all the desired symmetries included in the design obviates the need to include them by means of costly techniques such as data augmentation. We will discuss our PDE-based G-CNNs (PDE-G-CNNs) in a general homogeneous space setting while also going into the specifics of our primary case of interest: roto-translation equivariance. We solve the PDE of interest by a combination of linear group convolutions and non-linear morphological group convolutions with analytic kernel approximations that we underpin with formal theorems. Our kernel approximations allow for fast GPU-implementation of the PDE-solvers, we release our implementation with this article in the form of the LieTorch extension to PyTorch, available at https://gitlab.com/bsmetsjr/lietorch . Just like for linear convolution a morphological convolution is specified by a kernel that we train in our PDE-G-CNNs. In PDE-G-CNNs we do not use non-linearities such as max/min-pooling and ReLUs as they are already subsumed by morphological convolutions. We present a set of experiments to demonstrate the strength of the proposed PDE-G-CNNs in increasing the performance of deep learning based imaging applications with far fewer parameters than traditional CNNs.

arXiv comment27 pages, 18 figures. v2 changes: - mentioned KerCNNs - added section Generalization of G-CNNs - clarification that the experiments utilized automatic differentiation and SGD. v3 changes: - streamlined theoretical framework - formulation and proof Thm.1 & 2 - expanded experiments. v4 changes: typos in Prop.5 and (20) v5/6 changes: minor revision

References in corpus (24)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Rotation-invariant convolutional neural networks for galaxy morphology prediction
• Stable Architectures for Deep Neural Networks
• Rigid-Motion Scattering for Texture Classification
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• PDE-Net: Learning PDEs from Data
• Exploiting Cyclic Symmetry in Convolutional Neural Networks
• Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations
• General $E(2)$-Equivariant Steerable CNNs
• Collaborative Total Variation: A General Framework for Vectorial TV Models
• 3D G-CNNs for Pulmonary Nodule Detection
• Hypoelliptic diffusion and human vision: a semi-discrete new twist
• Geometrical Optical Illusion via Sub-Riemannian Geodesics in the Roto-Translation Group
• Template Matching via Densities on the Roto-Translation Group
• Numerical Approaches for Linear Left-invariant Diffusions on SE(2), their Comparison to Exact Solutions, and their Applications in Retinal Imaging
• A geometric model of multi-scale orientation preference maps via Gabor functions
• New Approximation of a Scale Space Kernel on SE(3) and Applications in Neuroimaging
• B-Spline CNNs on Lie Groups
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Fourier Transform on the Homogeneous Space of 3D Positions and Orientations for Exact Solutions to Linear Parabolic and (Hypo-)Elliptic PDEs
• KerCNNs: biologically inspired lateral connections for classification of corrupted images
• Probabilistic Numeric Convolutional Neural Networks
• Steerable Partial Differential Operators for Equivariant Neural Networks

Cited by in corpus (10)

• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Some open questions on morphological operators and representations in the deep learning era
• Probabilistic Numeric Convolutional Neural Networks
• Steerable Partial Differential Operators for Equivariant Neural Networks
• Translating Diffusion, Wavelets, and Regularisation into Residual Networks
• Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
• Finite Group Equivariant Neural Networks for Games
• ChebLieNet: Invariant Spectral Graph NNs Turned Equivariant by Riemannian Geometry on Lie Groups
• Orientation Scores should be a Piece of Cake