Multiple Classification with Split Learning

Abstract

Privacy issues were raised in the process of training deep learning in medical, mobility, and other fields. To solve this problem, we present privacy-preserving distributed deep learning method that allow clients to learn a variety of data without direct exposure. We divided a single deep learning architecture into a common extractor, a cloud model and a local classifier for the distributed learning. First, the common extractor, which is used by local clients, extracts secure features from the input data. The secure features also take the role that the cloud model can employ various task and diverse types of data. The feature contain the most important information that helps to proceed various task. Second, the cloud model including most parts of the whole training model gets the embedded features from the massive local clients, and performs most of deep learning operations which takes severe computing cost. After the operations in cloud model finished, outputs of the cloud model send back to local clients. Finally, the local classifier determined classification results and delivers the results to local clients. When clients train models, our model does not directly expose sensitive information to exterior network. During the test, the average performance improvement was 2.63% over the existing local training model. However, in a distributed environment, there is a possibility of inversion attack due to exposed features. For this reason, we experimented with the common extractor to prevent data restoration. The quality of restoration of the original image was tested by adjusting the depth of the common extractor. As a result, we found that the deeper the common extractor, the restoration score decreased to 89.74.

References in corpus (10)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• An Overview of Multi-Task Learning in Deep Neural Networks
• Federated Learning for Mobile Keyboard Prediction
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Multi-Task Learning as Multi-Objective Optimization
• Intra- and Inter-epoch Temporal Context Network (IITNet) Using Sub-epoch Features for Automatic Sleep Scoring on Raw Single-channel EEG
• Learning Multiple Tasks with Multilinear Relationship Networks
• Split learning for health: Distributed deep learning without sharing raw patient data
• Variational Federated Multi-Task Learning
• Deep Multi-task Representation Learning: A Tensor Factorisation Approach