On Learning Sets of Symmetric Elements

Abstract

Learning from unordered sets is a fundamental learning setup, recently attracting increasing attention. Research in this area has focused on the case where elements of the set are represented by feature vectors, and far less emphasis has been given to the common case where set elements themselves adhere to their own symmetries. That case is relevant to numerous applications, from deblurring image bursts to multi-view 3D shape recognition and reconstruction. In this paper, we present a principled approach to learning sets of general symmetric elements. We first characterize the space of linear layers that are equivariant both to element reordering and to the inherent symmetries of elements, like translation in the case of images. We further show that networks that are composed of these layers, called Deep Sets for Symmetric Elements (DSS) layers, are universal approximators of both invariant and equivariant functions, and that these networks are strictly more expressive than Siamese networks. DSS layers are also straightforward to implement. Finally, we show that they improve over existing set-learning architectures in a series of experiments with images, graphs, and point-clouds.

arXiv comment37th International Conference on Machine Learning, Vienna,2020, Outstanding paper award

References in corpus (27)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• Fast Graph Representation Learning with PyTorch Geometric
• Group Equivariant Convolutional Networks
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
• Order Matters: Sequence to sequence for sets
• Gauge Equivariant Convolutional Networks and the Icosahedral CNN
• Towards a Neural Statistician
• Universal Invariant and Equivariant Graph Neural Networks
• Equivariance Through Parameter-Sharing
• Invariant and Equivariant Graph Networks
• Covariant Compositional Networks For Learning Graphs
• 3D G-CNNs for Pulmonary Nodule Detection
• Deep Models of Interactions Across Sets
• N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials
• Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs
• On the equivalence between graph isomorphism testing and function approximation with GNNs
• Permutation-invariant Feature Restructuring for Correlation-aware Image Set-based Recognition
• A Simple Proof of the Universality of Invariant/Equivariant Graph Neural Networks
• RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks
• (Probably) Concave Graph Matching
• On Universal Equivariant Set Networks
• Equivariant Entity-Relationship Networks

Cited by in corpus (24)

• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding
• A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups
• Expressive Power of Invariant and Equivariant Graph Neural Networks
• On the Universality of Rotation Equivariant Point Cloud Networks
• Automatic Symmetry Discovery with Lie Algebra Convolutional Network
• Set2Graph: Learning Graphs From Sets
• Equivariant Subgraph Aggregation Networks
• Characterization Of Inpaint Residuals In Interferometric Measurements of the Epoch Of Reionization
• Fixed-Dimensional and Permutation Invariant State Representation of Autonomous Driving
• Deep Networks from the Principle of Rate Reduction
• Universal Approximation Theorem for Equivariant Maps by Group CNNs
• Equivariant Maps for Hierarchical Structures
• Trajectory Prediction using Equivariant Continuous Convolution
• On Adversarial Robustness of 3D Point Cloud Classification under Adaptive Attacks
• Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spaces
• Learning Prototype-oriented Set Representations for Meta-Learning
• Group Equivariant Stand-Alone Self-Attention For Vision
• Rotation-Invariant Autoencoders for Signals on Spheres
• About Explicit Variance Minimization: Training Neural Networks for Medical Imaging With Limited Data Annotations
• USCO-Solver: Solving Undetermined Stochastic Combinatorial Optimization Problems
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
• Equivariant Deep Dynamical Model for Motion Prediction
• Equivariant and Invariant Reynolds Networks
• Distribution-Based Invariant Deep Networks for Learning Meta-Features