36 citations · 124 across the 26 of their papers we have counts for
32 papers
TORQUE: Optimizing What (not) to Quantize Before and After Rotation
Ran Ben Basat, Michael Mitzenmacher, Shay Vargaftik
Uniform random rotations are an effective preprocessing step for quantization: they make normalized coordinate distributions approximately Gaussian, enabling the use of codebooks o…
Entropy-Constrained Adaptive Stochastic Quantization
Ran Ben Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher +1
Adaptive stochastic quantization (ASQ) is a recently introduced quantization approach that optimizes the Mean Squared Error (MSE) for a given input while preserving unbiasedness. I…
Quantizing With Randomized Hadamard Transforms: Efficient Heuristic Now Proven
Ran Ben-Basat, William Kuszmaul, Michael Mitzenmacher +2
Uniform random rotations (URRs) are a common preprocessing step in modern quantization approaches used for gradient compression, inference acceleration, KV-cache compression, model…
A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work
Ran Ben-Basat, Yaniv Ben-Itzhak, Gal Mendelson +3
This note clarifies the relationship between the recent TurboQuant work and the earlier DRIVE (NeurIPS 2021) and EDEN (ICML 2022) schemes. DRIVE is a 1-bit quantizer that EDEN exte…
DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reduce
Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +1
Multi-hop all-reduce is the de facto backbone of large model training. As the training scale increases, the network often becomes a bottleneck, motivating the reduction of the volu…
OptiNIC: A Resilient and Tail-Optimal RDMA NIC for Distributed ML Workloads
Ertza Warraich, Ali Imran, Annus Zulfiqar +3
As distributed machine learning (ML) workloads scale to thousands of GPUs connected by high-speed interconnects, tail latency in collective communication has become a major bottlen…
Reimagining RDMA Through the Lens of ML
Ertza Warraich, Ali Imran, Annus Zulfiqar +3
As distributed machine learning (ML) workloads scale to thousands of GPUs connected by ultra-high-speed inter-connects, tail latency in collective communication has emerged as a pr…
Counter Pools: Counter Representation for Efficient Stream Processing
Ran Ben Basat, Gil Einziger, Bilal Tyah +1
Due to the large data volume and number of distinct elements, space is often the bottleneck of many stream processing systems. The data structures used by these systems often consi…
HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
Zeyu Zhang, Haiying Shen, Shay Vargaftik +3
Disaggregated Large Language Model (LLM) inference has gained popularity as it separates the computation-intensive prefill stage from the memory-intensive decode stage, avoiding th…
Lucy: Think and Reason to Solve Text-to-SQL
Nina Narodytska, Shay Vargaftik
Large Language Models (LLMs) have made significant progress in assisting users to query databases in natural language. While LLM-based techniques provide state-of-the-art results o…
Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compression
Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +2
Gradient aggregation has long been identified as a major bottleneck in today's large-scale distributed machine learning training systems. One promising solution to mitigate such bo…
Optimal and Near-Optimal Adaptive Vector Quantization
Ran Ben-Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher +1
Quantization is a fundamental optimization for many machine-learning use cases, including compressing gradients, model weights and activations, and datasets. The most accurate form…
OptiReduce: Resilient and Tail-Optimal AllReduce for Distributed Deep Learning in the Cloud
Ertza Warraich, Omer Shabtai, Khalid Manaa +5
We present OptiReduce, a new collective-communication system for the cloud with bounded, predictable completion times for deep-learning jobs in the presence of varying computation…
THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression
Minghao Li, Ran Ben Basat, Shay Vargaftik +4
Deep neural networks (DNNs) are the de facto standard for essential use cases, such as image classification, computer vision, and natural language processing. As DNNs and datasets…
DoCoFL: Downlink Compression for Cross-Device Federated Learning
Ron Dorfman, Shay Vargaftik, Yaniv Ben-Itzhak +1
Many compression techniques have been proposed to reduce the communication overhead of Federated Learning training procedures. However, these are typically designed for compressing…
ScionFL: Efficient and Robust Secure Quantized Aggregation
Yaniv Ben-Itzhak, Helen Möllering, Benny Pinkas +7
Secure aggregation is commonly used in federated learning (FL) to alleviate privacy concerns related to the central aggregator seeing all parameter updates in the clear. Unfortunat…
IIsy: Practical In-Network Classification
Changgang Zheng, Zhaoqi Xiong, Thanh T Bui +6
The rat race between user-generated data and data-processing systems is currently won by data. The increased use of machine learning leads to further increase in processing require…
Automating In-Network Machine Learning
Changgang Zheng, Mingyuan Zang, Xinpeng Hong +4
Using programmable network devices to aid in-network machine learning has been the focus of significant research. However, most of the research was of a limited scope, providing a…
QUIC-FL: Quick Unbiased Compression for Federated Learning
Ran Ben Basat, Shay Vargaftik, Amit Portnoy +3
Distributed Mean Estimation (DME), in which clients communicate vectors to a parameter server that estimates their average, is a fundamental building block in communication-eff…
EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning
Shay Vargaftik, Ran Ben Basat, Amit Portnoy +3
Distributed Mean Estimation (DME) is a central building block in federated learning, where clients send local gradients to a parameter server for averaging and updating the model.…
Stochastic Coordination in Heterogeneous Load Balancing Systems
Guy Goren, Shay Vargaftik, Yoram Moses
Current-day data centers and high-volume cloud services employ a broad set of heterogeneous servers. In such settings, client requests typically arrive at multiple entry points, an…
DRIVE: One-bit Distributed Mean Estimation
Shay Vargaftik, Ran Ben Basat, Amit Portnoy +3
We consider the problem where clients transmit -dimensional real-valued vectors using bits each, in a manner that allows the receiver to approximately reconstruc…
On the Persistent-Idle Load Distribution Policy Under Batch Arrivals and Random Service Capacity
Rami Atar, Isaac Keslassy, Gal Mendelson +2
The Persistent-Idle (PI) load distribution policy was recently introduced as an appealing alternative to current low-communication load balancing techniques. In PI, servers only up…
SALSA: Self-Adjusting Lean Streaming Analytics
Ran Ben Basat, Gil Einziger, Michael Mitzenmacher +1
Counters are the fundamental building block of many data sketching schemes, which hash items to a small number of counters and account for collisions to provide good approximations…