Power Normalizations in Fine-grained Image, Few-shot Image and Graph Classification

Abstract

Power Normalizations (PN) are useful non-linear operators which tackle feature imbalances in classification problems. We study PNs in the deep learning setup via a novel PN layer pooling feature maps. Our layer combines the feature vectors and their respective spatial locations in the feature maps produced by the last convolutional layer of CNN into a positive definite matrix with second-order statistics to which PN operators are applied, forming so-called Second-order Pooling (SOP). As the main goal of this paper is to study Power Normalizations, we investigate the role and meaning of MaxExp and Gamma, two popular PN functions. To this end, we provide probabilistic interpretations of such element-wise operators and discover surrogates with well-behaved derivatives for end-to-end training. Furthermore, we look at the spectral applicability of MaxExp and Gamma by studying Spectral Power Normalizations (SPN). We show that SPN on the autocorrelation/covariance matrix and the Heat Diffusion Process (HDP) on a graph Laplacian matrix are closely related, thus sharing their properties. Such a finding leads us to the culmination of our work, a fast spectral MaxExp which is a variant of HDP for covariances/autocorrelation matrices. We evaluate our ideas on fine-grained recognition, scene recognition, and material classification, as well as in few-shot learning and graph classification.

arXiv commentAccepted by TPAMI, July 2020

References in corpus (10)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Fast Graph Representation Learning with PyTorch Geometric
• Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging
• Training Deeper Convolutional Networks with Deep Supervision
• Places205-VGGNet Models for Scene Recognition
• Anonymous Walk Embeddings
• A Riemannian Network for SPD Matrix Learning
• Graph Capsule Convolutional Neural Networks
• Capsule Neural Networks for Graph Classification using Explicit Tensorial Graph Representations

Cited by in corpus (13)

• LoGG3D-Net: Locally Guided Global Descriptor Learning for 3D Place Recognition
• COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
• Tensor Representations for Action Recognition
• Multi-level Second-order Few-shot Learning
• Convolutional Fine-Grained Classification with Self-Supervised Target Relation Regularization
• Self-supervising Action Recognition by Statistical Moment and Subspace Descriptors
• Graph-adaptive Rectified Linear Unit for Graph Neural Networks
• Few-shot Action Recognition with Permutation-invariant Attention
• Parsing Objects at a Finer Granularity: A Survey
• Rethinking Class Relations: Absolute-relative Supervised and Unsupervised Few-shot Learning
• ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring
• Sequential Random Network for Fine-grained Image Classification
• Graph Self-Supervised Learning with Learnable Structural and Positional Encodings