434 citations · 840 across the 16 of their papers we have counts for
16 papers
Baechi: Fast Device Placement of Machine Learning Graphs
Beomyeol Jeon, Linda Cai, Chirag Shetty +6
Machine Learning graphs (or models) can be challenging or impossible to train when either devices have limited memory, or models are large. To split the model across devices, learn…
Compressed Communication for Distributed Training: Adaptive Methods and System
Yuchen Zhong, Cong Xie, Shuai Zheng +1
Communication overhead severely hinders the scalability of distributed machine learning systems. Recently, there has been a growing interest in using gradient compression to reduce…
CSER: Communication-efficient SGD with Error Reset
Cong Xie, Shuai Zheng, Oluwasanmi Koyejo +3
The scalability of Distributed Stochastic Gradient Descent (SGD) is today limited by communication bottlenecks. We propose a novel SGD variant: Communication-efficient SGD with Err…
Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates
Cong Xie, Oluwasanmi Koyejo, Indranil Gupta +1
When scaling distributed training, the communication overhead is often the bottleneck. In this paper, we propose a novel SGD variant with reduced communication and adaptive learnin…
Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation
Cong Xie, Sanmi Koyejo, Indranil Gupta
Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning. The Byzantine model captures workers that behave arbitrarily,…
Zeno++: Robust Fully Asynchronous SGD
Cong Xie, Sanmi Koyejo, Indranil Gupta
We propose Zeno++, a new robust asynchronous Stochastic Gradient Descent~(SGD) procedure which tolerates Byzantine failures of the workers. In contrast to previous work, Zeno++ rem…
SLSGD: Secure and Efficient Distributed On-device Machine Learning
Cong Xie, Sanmi Koyejo, Indranil Gupta
We consider distributed on-device learning with limited communication and security requirements. We propose a new robust distributed optimization algorithm with efficient communica…
Asynchronous Federated Optimization
Cong Xie, Sanmi Koyejo, Indranil Gupta
Federated learning enables training on a massive number of edge devices. To improve flexibility and scalability, we propose a new asynchronous federated optimization algorithm. We…
Phocas: dimensional Byzantine-resilient stochastic gradient descent
Cong Xie, Oluwasanmi Koyejo, Indranil Gupta
We propose a novel robust aggregation rule for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily mani…
Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance
Cong Xie, Oluwasanmi Koyejo, Indranil Gupta
We present Zeno, a technique to make distributed machine learning, particularly Stochastic Gradient Descent (SGD), tolerant to an arbitrary number of faulty workers. Zeno generaliz…
Generalized Byzantine-tolerant SGD
Cong Xie, Oluwasanmi Koyejo, Indranil Gupta
We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily m…
Attack RMSE Leaderboard: An Introduction and Case Study
Cong Xie
In this manuscript, we briefly introduce several tricks to climb the leaderboards which use RMSE for evaluation without exploiting any training data.
Wishart Mechanism for Differentially Private Principal Components Analysis
Wuxuan Jiang, Cong Xie, Zhihua Zhang
We propose a new input perturbation mechanism for publishing a covariance matrix to achieve -differential privacy. Our mechanism uses a Wishart distribution to generate matr…
A New Relaxation Approach to Normalized Hypergraph Cut
Cong Xie, Wu-Jun Li, Zhihua Zhang
Normalized graph cut (NGC) has become a popular research topic due to its wide applications in a large variety of areas like machine learning and very large scale integration (VLSI…
S-PowerGraph: Streaming Graph Partitioning for Natural Graphs by Vertex-Cut
Cong Xie, Wu-Jun Li, Zhihua Zhang
One standard solution for analyzing large natural graphs is to adopt distributed computation on clusters. In distributed computation, graph partitioning (GP) methods assign the ver…
A Scalable and Extensible Framework for Superposition-Structured Models
Shenjian Zhao, Cong Xie, Zhihua Zhang
In many learning tasks, structural models usually lead to better interpretability and higher generalization performance. In recent years, however, the simple structural models such…