Convolutional Kernel Networks

Abstract

An important goal in visual recognition is to devise image representations that are invariant to particular transformations. In this paper, we address this goal with a new type of convolutional neural network (CNN) whose invariance is encoded by a reproducing kernel. Unlike traditional approaches where neural networks are learned either to represent data or for solving a classification task, our network learns to approximate the kernel feature map on training data. Such an approach enjoys several benefits over classical ones. First, by teaching CNNs to be invariant, we obtain simple network architectures that achieve a similar accuracy to more complex ones, while being easy to train and robust to overfitting. Second, we bridge a gap between the neural network literature and kernels, which are natural tools to model invariance. We evaluate our methodology on visual recognition tasks where CNNs have proven to perform well, e.g., digit recognition with the MNIST dataset, and the more challenging CIFAR-10 and STL-10 datasets, where our accuracy is competitive with the state of the art.

appears in Advances in Neural Information Processing Systems (NIPS), Dec 2014, Montreal, Canada, http://nips.cc

References in corpus (1)

• Learning Invariant Representations with Local Transformations

Cited by in corpus (22)

• Winner-Take-All Autoencoders
• Neural Kernels Without Tangents
• Delving Deeper into Anti-aliasing in ConvNets
• WaveCycleGAN2: Time-domain Neural Post-filter for Speech Waveform Generation
• Unsupervised Feature Learning with K-means and An Ensemble of Deep Convolutional Neural Networks for Medical Image Classification
• Wavelet Integrated CNNs for Noise-Robust Image Classification
• Fisher-Bures Adversary Graph Convolutional Networks
• When Kernel Methods meet Feature Learning: Log-Covariance Network for Action Recognition from Skeletal Data
• A Statistical Investigation of Long Memory in Language and Music
• Dataset Meta-Learning from Kernel Ridge-Regression
• Lesion Net -- Skin Lesion Segmentation Using Coordinate Convolution and Deep Residual Units
• Learning Representation for Mixed Data Types with a Nonlinear Deep Encoder-Decoder Framework
• Kernel-based Translations of Convolutional Networks
• The Unreasonable Effectiveness of Patches in Deep Convolutional Kernels Methods
• Deep Context-Aware Kernel Networks
• Higher-order Pooling of CNN Features via Kernel Linearization for Action Recognition
• A Neural Network Approach for Online Nonlinear Neyman-Pearson Classification
• Totally Deep Support Vector Machines
• End-to-end training of deep kernel map networks for image classification
• Deep Neural-Kernel Machines
• Do place cells dream of conditional probabilities? Learning Neural Nyström representations
• Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping