9 citations · 19 across the 12 of their papers we have counts for
15 papers
Entropy-Constrained Adaptive Stochastic Quantization
Ran Ben Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher +1
Adaptive stochastic quantization (ASQ) is a recently introduced quantization approach that optimizes the Mean Squared Error (MSE) for a given input while preserving unbiasedness. I…
ML-for-ML
Yutong Zhao, Noga H. Rotman, Gianni Antichi +1
AI training workloads are growing rapidly, making their time, energy, and infrastructure costs increasingly important. In shared cloud clusters, training and fine-tuning jobs compe…
Breaking the Barrier for the -Distinct Language
Ran Ben Basat
For integers , let be the set of words over of length at most in which no symbol is repeated. We present a nondeterministic finite automaton (NFA) of si…
DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reduce
Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +1
Multi-hop all-reduce is the de facto backbone of large model training. As the training scale increases, the network often becomes a bottleneck, motivating the reduction of the volu…
Sketch Disaggregation Across Time and Space
Jonatan Langlet, Peiqing Chen, Michael Mitzenmacher +3
Streaming analytics are essential in a large range of applications, including databases, networking, and machine learning. To optimize performance, practitioners are increasingly o…
Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compression
Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +2
Gradient aggregation has long been identified as a major bottleneck in today's large-scale distributed machine learning training systems. One promising solution to mitigate such bo…