A Communication-Efficient and Privacy-Aware Distributed Algorithm for Sparse PCA

Abstract

Sparse principal component analysis (PCA) improves interpretability of the classic PCA by introducing sparsity into the dimension-reduction process. Optimization models for sparse PCA, however, are generally non-convex, non-smooth and more difficult to solve, especially on large-scale datasets requiring distributed computation over a wide network. In this paper, we develop a distributed and centralized algorithm called DSSAL1 for sparse PCA that aims to achieve low communication overheads by adapting a newly proposed subspace-splitting strategy to accelerate convergence. Theoretically, convergence to stationary points is established for DSSAL1. Extensive numerical results show that DSSAL1 requires far fewer rounds of communication than state-of-the-art peer methods. In addition, we make the case that since messages exchanged in DSSAL1 are well-masked, the possibility of private-data leakage in DSSAL1 is much lower than in some other distributed algorithms.

References in corpus (6)

• A Second Order Non-Smooth Variational Model for Restoring Manifold-Valued Images
• A Dai-Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions
• Parallelizable Algorithms for Optimization Problems with Orthogonality Constraints
• A Linearly Convergent Algorithm for Distributed Principal Component Analysis
• Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function
• FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component Analysis

Cited by in corpus (1)

• Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function