Distributed Differentially-Private Algorithms for Matrix and Tensor Factorization

Abstract

In many signal processing and machine learning applications, datasets containing private information are held at different locations, requiring the development of distributed privacy-preserving algorithms. Tensor and matrix factorizations are key components of many processing pipelines. In the distributed setting, differentially private algorithms suffer because they introduce noise to guarantee privacy. This paper designs new and improved distributed and differentially private algorithms for two popular matrix and tensor factorization methods: principal component analysis (PCA) and orthogonal tensor decomposition (OTD). The new algorithms employ a correlated noise design scheme to alleviate the effects of noise and can achieve the same noise level as the centralized scenario. Experiments on synthetic and real data illustrate the regimes in which the correlated noise allows performance matching with the centralized setting, outperforming previous methods and demonstrating that meaningful utility is possible while guaranteeing differential privacy.

arXiv comment39 pages, in review for publication

References in corpus (8)

• Improved Distributed Principal Component Analysis
• A Distributed k-Secure Sum Protocol for Secure Multi-Party Computations
• Distributed Principal Component Analysis for Wireless Sensor Networks
• Lower Bounds for Locally Private Estimation via Communication Complexity
• Symmetric Orthogonal Tensor Decomposition is Trivial
• Online and Differentially-Private Tensor Decomposition
• The Podium Mechanism: Improving on the Laplace and Staircase Mechanisms
• An end-to-end Differentially Private Latent Dirichlet Allocation Using a Spectral Algorithm

Cited by in corpus (8)

• When Machine Learning Meets Privacy: A Survey and Outlook
• FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component Analysis
• Reviewing and Improving the Gaussian Mechanism for Differential Privacy
• Distributed Differentially Private Computation of Functions with Correlated Noise
• Columnwise Element Selection for Computationally Efficient Nonnegative Coupled Matrix Tensor Factorization
• Improved Differentially Private Decentralized Source Separation for fMRI Data
• Federated Unlearning
• Protecting Big Data Privacy Using Randomized Tensor Network Decomposition and Dispersed Tensor Computation