63 citations · 177 across the 12 of their papers we have counts for
12 papers
Amazon SageMaker Model Parallelism: A General and Flexible Framework for Large Model Training
Can Karakus, Rahul Huilgol, Fei Wu +7
With deep learning models rapidly growing in size, systems-level solutions for large-model training are required. We present Amazon SageMaker model parallelism, a software library…
Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations
Debraj Basu, Deepesh Data, Can Karakus +1
Communication bottleneck has been identified as a significant issue in distributed optimization of large-scale learning models. Recently, several approaches to mitigate this proble…
Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models
Derya Cavdar, Valeriu Codreanu, Can Karakus +11
Neural machine translation - using neural networks to translate human language - is an area of active research exploring new neuron types and network topologies with the goal of dr…
Differentially Private Consensus-Based Distributed Optimization
Mehrdad Showkatbakhsh, Can Karakus, Suhas Diggavi
Data privacy is an important concern in learning, when datasets contain sensitive information about individuals. This paper considers consensus-based distributed optimization under…
Privacy-Utility Trade-off of Linear Regression under Random Projections and Additive Noise
Mehrdad Showkatbakhsh, Can Karakus, Suhas Diggavi
Data privacy is an important concern in machine learning, and is fundamentally at odds with the task of training useful learning models, which typically require the acquisition of…
Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning
Can Karakus, Yifan Sun, Suhas Diggavi +1
Performance of distributed optimization and learning systems is bottlenecked by "straggler" nodes and slow communication links, which significantly delay computation. We propose a…