Scaling the Scattering Transform: Deep Hybrid Networks

Abstract

We use the scattering network as a generic and fixed ini-tialization of the first layers of a supervised hybrid deep network. We show that early layers do not necessarily need to be learned, providing the best results to-date with pre-defined representations while being competitive with Deep CNNs. Using a shallow cascade of 1 x 1 convolutions, which encodes scattering coefficients that correspond to spatial windows of very small sizes, permits to obtain AlexNet accuracy on the imagenet ILSVRC2012. We demonstrate that this local encoding explicitly learns invariance w.r.t. rotations. Combining scattering networks with a modern ResNet, we achieve a single-crop top 5 error of 11.4% on imagenet ILSVRC2012, comparable to the Resnet-18 architecture, while utilizing only 10 layers. We also find that hybrid architectures can yield excellent performance in the small sample regime, exceeding their end-to-end counterparts, through their ability to incorporate geometrical priors. We demonstrate this on subsets of the CIFAR-10 dataset and on the STL-10 dataset.

References in corpus (6)

• Striving for Simplicity: The All Convolutional Net
• Wide Residual Networks
• What makes ImageNet good for transfer learning?
• Deep unsupervised learning through spatial contrasting
• Inverse Problems with Invariant Multiscale Statistics
• Building a Regular Decision Boundary with Deep Networks

Cited by in corpus (9)

• Unsupervised Representation Learning by Predicting Image Rotations
• A critical analysis of self-supervision, or what we can learn from a single image
• A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth
• Deep Learning in the Wavelet Domain
• Harmonic Networks: Integrating Spectral Information into CNNs
• Wavelet-Based Dual-Branch Network for Image Demoireing
• Generating superpixels using deep image representations
• Skeleton-Based Action Recognition with Synchronous Local and Non-local Spatio-temporal Learning and Frequency Attention
• Invariant Integration in Deep Convolutional Feature Space