Local Group Invariant Representations via Orbit Embeddings
arXiv:1612.01988
Abstract
Invariance to nuisance transformations is one of the desirable properties of effective representations. We consider transformations that form a \emph{group} and propose an approach based on kernel methods to derive local group invariant representations. Locality is achieved by defining a suitable probability distribution over the group which in turn induces distributions in the input feature space. We learn a decision function over these distributions by appealing to the powerful framework of kernel methods and generate local invariant random feature maps via kernel approximations. We show uniform convergence bounds for kernel approximation and provide excess risk bounds for learning with these features. We evaluate our method on three real datasets, including Rotated MNIST and CIFAR-10, and observe that it outperforms competing kernel based approaches. The proposed method also outperforms deep CNN on Rotated-MNIST and performs comparably to the recently proposed group-equivariant CNN.
arXiv commentAISTATS 2017 accepted version including appendix, 18 pages, 1 figure
References in corpus (10)
- Rotation-invariant convolutional neural networks for galaxy morphology prediction
- Kernel Mean Embedding of Distributions: A Review and Beyond
- Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
- Scalable Kernel Methods via Doubly Stochastic Gradients
- Unsupervised Learning of Invariant Representations in Hierarchical Architectures
- Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels
- How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets
- Recycling Randomness with Structure for Sublinear time Kernel Expansions
- Structured adaptive and random spinners for fast machine learning computations
- Learning from Conditional Distributions via Dual Embeddings
Cited by in corpus (6)
- Semi-supervised Learning with GANs: Manifold Invariance with Improved Inference
- Variational Inference of Disentangled Latent Concepts from Unlabeled Observations
- Max-Margin Invariant Features from Transformed Unlabeled Data
- On the effect of pooling on the geometry of representations
- VolterraNet: A higher order convolutional network with group equivariance for homogeneous manifolds
- Discriminative Learning of Similarity and Group Equivariant Representations