Feature-Distributed SVRG for High-Dimensional Linear Classification

Abstract

Linear classification has been widely used in many high-dimensional applications like text classification. To perform linear classification for large-scale tasks, we often need to design distributed learning methods on a cluster of multiple machines. In this paper, we propose a new distributed learning method, called feature-distributed stochastic variance reduced gradient (FD-SVRG) for high-dimensional linear classification. Unlike most existing distributed learning methods which are instance-distributed, FD-SVRG is feature-distributed. FD-SVRG has lower communication cost than other instance-distributed methods when the data dimensionality is larger than the number of data instances. Experimental results on real data demonstrate that FD-SVRG can outperform other state-of-the-art distributed methods for high-dimensional linear classification in terms of both communication cost and wall-clock time, when the dimensionality is larger than the number of instances in training data.

References in corpus (1)

• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives

Cited by in corpus (8)

• Vertical Federated Learning: Concepts, Advances and Challenges
• Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
• Achieving Model Fairness in Vertical Federated Learning
• A Vertical Federated Learning Framework for Horizontally Partitioned Labels
• From Federated Learning to Federated Neural Architecture Search: A Survey
• Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating
• Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm
• Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data