6 citations · 7 across the 20 of their papers we have counts for
7 papers · 1 filter
Fast Inference for Augmented Large Language Models
Rana Shahout, Cong Liang, Shiji Xin +4
Augmented Large Language Models (LLMs) enhance the capabilities of standalone LLMs by integrating external data sources through API calls. In interactive LLM applications, efficien…
Don't Stop Me Now: Embedding Based Scheduling for LLMs
Rana Shahout, Eran Malach, Chunwei Liu +3
Efficient scheduling is crucial for interactive Large Language Model (LLM) applications, where low request completion time directly impacts user engagement. Size-based scheduling a…
Learning-Augmented Frequency Estimation in Sliding Windows
Rana Shahout, Ibrahim Sabek, Michael Mitzenmacher
We show how to utilize machine learning approaches to improve sliding window algorithms for approximate frequency estimation problems, under the ``algorithms with predictions'' fra…
Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compression
Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +2
Gradient aggregation has long been identified as a major bottleneck in today's large-scale distributed machine learning training systems. One promising solution to mitigate such bo…
Learning-Based Heavy Hitters and Flow Frequency Estimation in Streams
Rana Shahout, Michael Mitzenmacher
Identifying heavy hitters and estimating the frequencies of flows are fundamental tasks in various network domains. Existing approaches to this challenge can broadly be categorized…
SkipPredict: When to Invest in Predictions for Scheduling
Rana Shahout, Michael Mitzenmacher
In light of recent work on scheduling with predicted job sizes, we consider the effect of the cost of predictions in queueing systems, removing the assumption in prior research tha…
Optimal and Near-Optimal Adaptive Vector Quantization
Ran Ben-Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher +1
Quantization is a fundamental optimization for many machine-learning use cases, including compressing gradients, model weights and activations, and datasets. The most accurate form…