A Riemannian Network for SPD Matrix Learning

Abstract

Symmetric Positive Definite (SPD) matrix learning methods have become popular in many image and video processing tasks, thanks to their ability to learn appropriate statistical representations while respecting Riemannian geometry of underlying SPD manifolds. In this paper we build a Riemannian network architecture to open up a new direction of SPD matrix non-linear learning in a deep model. In particular, we devise bilinear mapping layers to transform input SPD matrices to more desirable SPD matrices, exploit eigenvalue rectification layers to apply a non-linear activation function to the new SPD matrices, and design an eigenvalue logarithm layer to perform Riemannian computing on the resulting SPD matrices for regular output layers. For training the proposed deep network, we exploit a new backpropagation with a variant of stochastic gradient descent on Stiefel manifolds to update the structured connection weights and the involved SPD matrix data. We show through experiments that the proposed SPD matrix network can be simply trained and outperform existing SPD matrix learning and state-of-the-art methods in three typical visual classification tasks.

Revised arXiv version, AAAI-17 camera-ready

Cited by in corpus (34)

• Power Normalizations in Fine-grained Image, Few-shot Image and Graph Classification
• Action-Attending Graphic Neural Network
• Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition
• Recurrent Embedding Aggregation Network for Video Face Recognition
• Algorithms for the computation of the matrix logarithm based on the double exponential formula
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Shared Subspace Models for Multi-Group Covariance Estimation
• When Kernel Methods meet Feature Learning: Log-Covariance Network for Action Recognition from Skeletal Data
• A Two-stream Neural Network for Pose-based Hand Gesture Recognition
• On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometry
• Some open questions on morphological operators and representations in the deep learning era
• Dependency-aware Attention Control for Unconstrained Face Recognition with Image Sets
• HAN: An Efficient Hierarchical Self-Attention Network for Skeleton-Based Gesture Recognition
• Divide-and-Conquer Adversarial Learning for High-Resolution Image and Video Enhancement
• MVC-Net: A Convolutional Neural Network Architecture for Manifold-Valued Images With Applications
• Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach
• Deep and Shallow Covariance Feature Quantization for 3D Facial Expression Recognition
• Constructing the Matrix Multilayer Perceptron and its Application to the VAE
• Dilated Convolutional Neural Networks for Sequential Manifold-valued Data
• Manifold-valued Image Generation with Wasserstein Generative Adversarial Nets
• Skeleton-Based Hand Gesture Recognition by Learning SPD Matrices with Neural Networks
• Skeleton-based Hand-Gesture Recognition with Lightweight Graph Convolutional Networks
• Accelerated Algorithms for Convex and Non-Convex Optimization on Manifolds
• Flow-based Generative Models for Learning Manifold to Manifold Mappings
• A Matrix Autoencoder Framework to Align the Functional and Structural Connectivity Manifolds as Guided by Behavioral Phenotypes
• Feature Super-Resolution Based Facial Expression Recognition for Multi-scale Low-Resolution Faces
• GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition
• Learning Spectral Transform Network on 3D Surface for Non-rigid Shape Analysis
• Multiple Manifolds Metric Learning with Application to Image Set Classification
• Riemannian joint dimensionality reduction and dictionary learning on symmetric positive definite manifold
• Eigendecomposition-Free Training of Deep Networks for Linear Least-Square Problems
• Entangled Kernels -- Beyond Separability
• More About Covariance Descriptors for Image Set Coding: Log-Euclidean Framework based Kernel Matrix Representation
• Optimization on Product Submanifolds of Convolution Kernels