HyPHEN: A Hybrid Packing Method and Optimizations for Homomorphic Encryption-Based Neural Networks

Abstract

Convolutional neural network (CNN) inference using fully homomorphic encryption (FHE) is a promising private inference (PI) solution due to the capability of FHE that enables offloading the whole computation process to the server while protecting the privacy of sensitive user data. Prior FHE-based CNN (HCNN) work has demonstrated the feasibility of constructing deep neural network architectures such as ResNet using FHE. Despite these advancements, HCNN still faces significant challenges in practicality due to the high computational and memory overhead. To overcome these limitations, we present HyPHEN, a deep HCNN construction that incorporates novel convolution algorithms (RAConv and CAConv), data packing methods (2D gap packing and PRCR scheme), and optimization techniques tailored to HCNN construction. Such enhancements enable HyPHEN to substantially reduce the memory footprint and the number of expensive homomorphic operations, such as ciphertext rotation and bootstrapping. As a result, HyPHEN brings the latency of HCNN CIFAR-10 inference down to a practical level at 1.4 seconds (ResNet-20) and demonstrates HCNN ImageNet inference for the first time at 14.7 seconds (ResNet-18).

arXiv comment15 pages, 12 figures

References in corpus (10)

• Distilling the Knowledge in a Neural Network
• CrypTFlow2: Practical 2-Party Secure Inference
• BTS: An Accelerator for Bootstrappable Fully Homomorphic Encryption
• ARK: Fully Homomorphic Encryption Accelerator with Runtime Data Generation and Inter-Operation Key Reuse
• EVA: An Encrypted Vector Arithmetic Language and Compiler for Efficient Homomorphic Computation
• HEAAN Demystified: Accelerating Fully Homomorphic Encryption Through Architecture-centric Analysis and Optimization
• Accelerating Number Theoretic Transformations for Bootstrappable Homomorphic Encryption on GPUs
• HeLayers: A Tile Tensors Framework for Large Neural Networks on Encrypted Data
• Precise Approximation of Convolutional Neural Networks for Homomorphically Encrypted Data
• AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference

Cited by in corpus (3)

• NeuJeans: Private Neural Network Inference with Joint Optimization of Convolution and FHE Bootstrapping
• Secure Outsourced Decryption for FHE-based Privacy-preserving Cloud Computing
• Cheddar: A Swift Fully Homomorphic Encryption Library Designed for GPU Architectures