Learning Invariant Representations with Local Transformations

Abstract

Learning invariant representations is an important problem in machine learning and pattern recognition. In this paper, we present a novel framework of transformation-invariant feature learning by incorporating linear transformations into the feature learning algorithms. For example, we present the transformation-invariant restricted Boltzmann machine that compactly represents data by its weights and their transformations, which achieves invariance of the feature representation via probabilistic max pooling. In addition, we show that our transformation-invariant feature learning framework can also be extended to other unsupervised learning methods, such as autoencoders or sparse coding. We evaluate our method on several image classification benchmark datasets, such as MNIST variations, CIFAR-10, and STL-10, and show competitive or superior classification performance when compared to the state-of-the-art. Furthermore, our method achieves state-of-the-art performance on phone classification tasks with the TIMIT dataset, which demonstrates wide applicability of our proposed algorithms to other domains.

Appears in Proceedings of the 29th International Conference on Machine Learning (ICML 2012)

References in corpus (1)

• High-Performance Neural Networks for Visual Object Classification

Cited by in corpus (21)

• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Group Equivariant Convolutional Networks
• Rotation equivariant vector field networks
• Convolutional Kernel Networks
• Exploiting Cyclic Symmetry in Convolutional Neural Networks
• Locally Scale-Invariant Convolutional Neural Networks
• CyCNN: A Rotation Invariant CNN using Polar Mapping and Cylindrical Convolution Layers
• Self-learning projective quantum Monte Carlo simulations guided by restricted Boltzmann machines
• A Study of the Generalizability of Self-Supervised Representations
• Simulating disordered quantum systems via dense and sparse restricted Boltzmann machines
• Unsupervised learning from videos using temporal coherency deep networks
• "Mental Rotation" by Optimizing Transforming Distance
• Self-Supervised Multi-View Learning via Auto-Encoding 3D Transformations
• Auto-Classifier: A Robust Defect Detector Based on an AutoML Head
• GraphTER: Unsupervised Learning of Graph Transformation Equivariant Representations via Auto-Encoding Node-wise Transformations
• Learning image representations tied to ego-motion
• Learnable Gabor modulated complex-valued networks for orientation robustness
• Euclidean Invariant Recognition of 2D Shapes Using Histograms of Magnitudes of Local Fourier-Mellin Descriptors
• Affine Disentangled GAN for Interpretable and Robust AV Perception
• Attn-HybridNet: Improving Discriminability of Hybrid Features with Attention Fusion
• Transform-Invariant Convolutional Neural Networks for Image Classification and Search