Proof of Learning (PoLe): Empowering Machine Learning with Consensus Building on Blockchains

Abstract

The progress of deep learning (DL), especially the recent development of automatic design of networks, has brought unprecedented performance gains at heavy computational cost. On the other hand, blockchain systems routinely perform a huge amount of computation that does not achieve practical purposes in order to build Proof-of-Work (PoW) consensus from decentralized participants. In this paper, we propose a new consensus mechanism, Proof of Learning (PoLe), which directs the computation spent for consensus toward optimization of neural networks (NN). In our mechanism, the training/testing data are released to the entire blockchain network (BCN) and the consensus nodes train NN models on the data, which serves as the proof of learning. When the consensus on the BCN considers a NN model to be valid, a new block is appended to the blockchain. We experimentally compare the PoLe protocol with Proof of Work (PoW) and show that PoLe can achieve a more stable block generation rate, which leads to more efficient transaction processing. We also introduce a novel cheating prevention mechanism, Secure Mapping Layer (SML), which can be straightforwardly implemented as a linear NN layer. Empirical evaluation shows that SML can detect cheating nodes at small cost to the predictive performance.

11 pages,8 figures

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• A Survey on Security and Privacy Issues of Bitcoin
• Billion-scale semi-supervised learning for image classification
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• The Evolved Transformer
• Blockchain platform with proof-of-work based on analog Hamiltonian optimisers
• Searching for Stage-wise Neural Graphs In the Limit