Image and Model Transformation with Secret Key for Vision Transformer

Abstract

In this paper, we propose a combined use of transformed images and vision transformer (ViT) models transformed with a secret key. We show for the first time that models trained with plain images can be directly transformed to models trained with encrypted images on the basis of the ViT architecture, and the performance of the transformed models is the same as models trained with plain images when using test images encrypted with the key. In addition, the proposed scheme does not require any specially prepared data for training models or network modification, so it also allows us to easily update the secret key. In an experiment, the effectiveness of the proposed scheme is evaluated in terms of performance degradation and model protection performance in an image classification task on the CIFAR-10 dataset.

10 pages, 5 figures

References in corpus (9)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• MLP-Mixer: An all-MLP Architecture for Vision
• CycleMLP: A MLP-like Architecture for Dense Prediction
• P3: Toward Privacy-Preserving Photo Sharing
• Adversarial Machine Learning -- Industry Perspectives
• A Privacy-Preserving Machine Learning Scheme Using EtC Images
• A Reversible Data Hiding Method in Compressible Encrypted Images
• Block shuffling learning for Deepfake Detection

Cited by in corpus (1)

• Keys in the Weights: Transformer Authentication Using Model-Bound Latent Representations