Convolutional Neural Networks with Transformed Input based on Robust Tensor Network Decomposition

Abstract

Tensor network decomposition, originated from quantum physics to model entangled many-particle quantum systems, turns out to be a promising mathematical technique to efficiently represent and process big data in parsimonious manner. In this study, we show that tensor networks can systematically partition structured data, e.g. color images, for distributed storage and communication in privacy-preserving manner. Leveraging the sea of big data and metadata privacy, empirical results show that neighbouring subtensors with implicit information stored in tensor network formats cannot be identified for data reconstruction. This technique complements the existing encryption and randomization techniques which store explicit data representation at one place and highly susceptible to adversarial attacks such as side-channel attacks and de-anonymization. Furthermore, we propose a theory for adversarial examples that mislead convolutional neural networks to misclassification using subspace analysis based on singular value decomposition (SVD). The theory is extended to analyze higher-order tensors using tensor-train SVD (TT-SVD); it helps to explain the level of susceptibility of different datasets to adversarial attacks, the structural similarity of different adversarial attacks including global and localized attacks, and the efficacy of different adversarial defenses based on input transformation. An efficient and adaptive algorithm based on robust TT-SVD is then developed to detect strong and static adversarial attacks.

References in corpus (9)

• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Tensor Decomposition for Signal Processing and Machine Learning
• Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
• A study of the effect of JPG compression on adversarial images
• Spatially Transformed Adversarial Examples
• Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods
• Compressing Recurrent Neural Network with Tensor Train
• Deep Multi-task Representation Learning: A Tensor Factorisation Approach
• Robustness of Rotation-Equivariant Networks to Adversarial Perturbations