6 citations · 7 across the 20 of their papers we have counts for
21 papers
Entropy-Constrained Adaptive Stochastic Quantization
Ran Ben Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher +1
Adaptive stochastic quantization (ASQ) is a recently introduced quantization approach that optimizes the Mean Squared Error (MSE) for a given input while preserving unbiasedness. I…
On Topology's Role in ML Training Performance
Sarah McClure, Tegan Wilson, Brad Karp +4
Modern machine learning training workloads run on large-scale networks of compute accelerators. The networks commonly deployed in these systems are typically variations of two basi…
Context Compaction Theory
Hayder Tirmazi, Sam Markelon, Allison Bishop +1
Large Language Models (LLMs) have a bounded context window. The context window is the maximum input size an LLM can consume for a single inference. AI agents rely on a process call…
SVR-MAD: A Bayesian-Inspired Framework for Posterior-Guided Multi-Agent Debate
Weifan Jiang, Rana Shahout, Minghao Li +4
Multi-Agent Debate (MAD) improves LLM-agent accuracy but suffers from rapid context growth, limiting scalability in larger multi-agent settings. Existing methods prune low-utility…
Orla: A Library for Serving LLM-Based Multi-Agent Systems
Rana Shahout, Hayder Tirmazi, Minlan Yu +1
We introduce Orla, a library for constructing and running LLM-based agentic systems. Modern agentic applications consist of workflows that combine multiple LLM inference steps, too…
DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reduce
Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +1
Multi-hop all-reduce is the de facto backbone of large model training. As the training scale increases, the network often becomes a bottleneck, motivating the reduction of the volu…
Mixed updating in structured populations
David A. Brewster, Yichen Huang, Michael Mitzenmacher +1
Evolutionary graph theory (EGT) studies the effect of population structure on evolutionary dynamics. The vertices of the graph represent the individuals. The edges denote inter…
The Mixed Birth-death/death-Birth Moran Process
David A. Brewster, Yichen Huang, Michael Mitzenmacher +1
We study evolutionary dynamics on graphs in which each step consists of one birth and one death, also known as the Moran processes. There are two types of individuals: residents wi…
From Score Distributions to Balance: Plug-and-Play Mixture-of-Experts Routing
Rana Shahout, Colin Cai, Yilun Du +2
Mixture-of-Experts (MoE) models can scale parameter capacity by routing each token to a subset of experts through a learned gate function. While conditional routing reduces trainin…
Intra-request branch orchestration for efficient LLM reasoning
Weifan Jiang, Rana Shahout, Yilun Du +2
Large Language Models (LLMs) increasingly rely on inference-time reasoning algorithms such as chain-of-thought and multi-branch reasoning to improve accuracy on complex tasks. Thes…
Sketch Disaggregation Across Time and Space
Jonatan Langlet, Peiqing Chen, Michael Mitzenmacher +3
Streaming analytics are essential in a large range of applications, including databases, networking, and machine learning. To optimize performance, practitioners are increasingly o…
Queueing, Predictions, and LLMs: Challenges and Open Problems
Michael Mitzenmacher, Rana Shahout
Queueing systems present many opportunities for applying machine-learning predictions, such as estimated service times, to improve system performance. This integration raises numer…
HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
Zeyu Zhang, Haiying Shen, Shay Vargaftik +3
Disaggregated Large Language Model (LLM) inference has gained popularity as it separates the computation-intensive prefill stage from the memory-intensive decode stage, avoiding th…
Fast Inference for Augmented Large Language Models
Rana Shahout, Cong Liang, Shiji Xin +4
Augmented Large Language Models (LLMs) enhance the capabilities of standalone LLMs by integrating external data sources through API calls. In interactive LLM applications, efficien…
Don't Stop Me Now: Embedding Based Scheduling for LLMs
Rana Shahout, Eran Malach, Chunwei Liu +3
Efficient scheduling is crucial for interactive Large Language Model (LLM) applications, where low request completion time directly impacts user engagement. Size-based scheduling a…
Learning-Augmented Frequency Estimation in Sliding Windows
Rana Shahout, Ibrahim Sabek, Michael Mitzenmacher
We show how to utilize machine learning approaches to improve sliding window algorithms for approximate frequency estimation problems, under the ``algorithms with predictions'' fra…
Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compression
Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +2
Gradient aggregation has long been identified as a major bottleneck in today's large-scale distributed machine learning training systems. One promising solution to mitigate such bo…
Learning-Based Heavy Hitters and Flow Frequency Estimation in Streams
Rana Shahout, Michael Mitzenmacher
Identifying heavy hitters and estimating the frequencies of flows are fundamental tasks in various network domains. Existing approaches to this challenge can broadly be categorized…
SkipPredict: When to Invest in Predictions for Scheduling
Rana Shahout, Michael Mitzenmacher
In light of recent work on scheduling with predicted job sizes, we consider the effect of the cost of predictions in queueing systems, removing the assumption in prior research tha…
Optimal and Near-Optimal Adaptive Vector Quantization
Ran Ben-Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher +1
Quantization is a fundamental optimization for many machine-learning use cases, including compressing gradients, model weights and activations, and datasets. The most accurate form…
How to send a real number using a single bit (and some shared randomness)
Ran Ben-Basat, Michael Mitzenmacher, Shay Vargaftik
We consider the fundamental problem of communicating an estimate of a real number using a single bit. A sender that knows chooses a value to transmi…