Exploiting Redundancy: Separable Group Convolutional Networks on Lie Groups

Abstract

Group convolutional neural networks (G-CNNs) have been shown to increase parameter efficiency and model accuracy by incorporating geometric inductive biases. In this work, we investigate the properties of representations learned by regular G-CNNs, and show considerable parameter redundancy in group convolution kernels. This finding motivates further weight-tying by sharing convolution kernels over subgroups. To this end, we introduce convolution kernels that are separable over the subgroup and channel dimensions. In order to obtain equivariance to arbitrary affine Lie groups we provide a continuous parameterisation of separable convolution kernels. We evaluate our approach across several vision datasets, and show that our weight sharing leads to improved performance and computational efficiency. In many settings, separable G-CNNs outperform their non-separable counterpart, while only using a fraction of their training time. In addition, thanks to the increase in computational efficiency, we are able to implement G-CNNs equivariant to the $\mathrm{Sim}(2)$ group; the group of dilations, rotations and translations. $\mathrm{Sim}(2)$-equivariance further improves performance on all tasks considered.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Rigid-Motion Scattering for Texture Classification
• 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
• Deep learning of multi-element abundances from high-resolution spectroscopic data
• 3D G-CNNs for Pulmonary Nodule Detection
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
• Deep Scale-spaces: Equivariance Over Scale
• Scale-Equivariant Steerable Networks
• Sample Efficient Semantic Segmentation using Rotation Equivariant Convolutional Networks
• B-Spline CNNs on Lie Groups
• CKConv: Continuous Kernel Convolution For Sequential Data