Communication Efficient Generalized Tensor Factorization for Decentralized Healthcare Networks

Abstract

Tensor factorization has been proved as an efficient unsupervised learning approach for health data analysis, especially for computational phenotyping, where the high-dimensional Electronic Health Records (EHRs) with patients' history of medical procedures, medications, diagnosis, lab tests, etc., are converted to meaningful and interpretable medical concepts. Federated tensor factorization distributes the tensor computation to multiple workers under the coordination of a central server, which enables jointly learning the phenotypes across multiple hospitals while preserving the privacy of the patient information. However, existing federated tensor factorization algorithms encounter the single-point-failure issue with the involvement of the central server, which is not only easily exposed to external attacks but also limits the number of clients sharing information with the server under restricted uplink bandwidth. In this paper, we propose CiderTF, a communication-efficient decentralized generalized tensor factorization, which reduces the uplink communication cost by leveraging a four-level communication reduction strategy designed for a generalized tensor factorization, which has the flexibility of modeling different tensor distribution with multiple kinds of loss functions. Experiments on two real-world EHR datasets demonstrate that CiderTF achieves comparable convergence with a communication reduction up to 99.99%.

arXiv commentShort version accepted to IEEE ICDM 2021

References in corpus (15)

• Scalable Tensor Factorizations for Incomplete Data
• Sparsified SGD with Memory
• Don't Use Large Mini-Batches, Use Local SGD
• Communication Compression for Decentralized Training
• D${}^2$: Decentralized Training over Decentralized Data
• A Unified Theory of Decentralized SGD with Changing Topology and Local Updates
• Error Feedback Fixes SignSGD and other Gradient Compression Schemes
• Generalized Canonical Polyadic Tensor Decomposition
• Federated Tensor Factorization for Computational Phenotyping
• DFacTo: Distributed Factorization of Tensors
• The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication
• Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback
• Moniqua: Modulo Quantized Communication in Decentralized SGD
• Global Convergence of Block Coordinate Descent in Deep Learning
• Blockchain Assisted Decentralized Federated Learning (BLADE-FL) with Lazy Clients