Quaternion Equivariant Capsule Networks for 3D Point Clouds

Abstract

We present a 3D capsule module for processing point clouds that is equivariant to 3D rotations and translations, as well as invariant to permutations of the input points. The operator receives a sparse set of local reference frames, computed from an input point cloud and establishes end-to-end transformation equivariance through a novel dynamic routing procedure on quaternions. Further, we theoretically connect dynamic routing between capsules to the well-known Weiszfeld algorithm, a scheme for solving \emph{iterative re-weighted least squares} (IRLS) problems with provable convergence properties. It is shown that such group dynamic routing can be interpreted as robust IRLS rotation averaging on capsule votes, where information is routed based on the final inlier scores. Based on our operator, we build a capsule network that disentangles geometry from pose, paving the way for more informative descriptors and a structured latent space. Our architecture allows joint object classification and orientation estimation without explicit supervision of rotations. We validate our algorithm empirically on common benchmark datasets.

arXiv commentOral Presentation at ECCV 2020. Find our video under: https://youtu.be/LHh56snwhTA. We release our sources at: http://tolgabirdal.github.io/qecnetworks

References in corpus (13)

• Group Equivariant Convolutional Networks
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
• Rotation equivariant vector field networks
• Riemannian Adaptive Optimization Methods
• Group Equivariant Capsule Networks
• Spherical CNNs on Unstructured Grids
• Iterative Transformer Network for 3D Point Cloud
• Bayesian Pose Graph Optimization via Bingham Distributions and Tempered Geodesic MCMC
• Geometric Capsule Autoencoders for 3D Point Clouds
• Cross-Domain 3D Equivariant Image Embeddings
• Equivariant neural networks and equivarification

Cited by in corpus (4)

• On the Universality of Rotation Equivariant Point Cloud Networks
• PaRot: Patch-Wise Rotation-Invariant Network via Feature Disentanglement and Pose Restoration
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Continuous Geodesic Convolutions for Learning on 3D Shapes