2 citations · 3 across the 16 of their papers we have counts for
18 papers
Jumping the Line: Exploiting Length Predictions in LLM Scheduling
Yuyang Dai, Rana Shahout, Mahmood Sharif
Efficient request scheduling is increasingly important for reducing completion time in large language model (LLM) serving. Size-based policies such as Shortest Job First prioritize…
SVR-MAD: A Bayesian-Inspired Framework for Posterior-Guided Multi-Agent Debate
Weifan Jiang, Rana Shahout, Minghao Li +4
Multi-Agent Debate (MAD) improves LLM-agent accuracy but suffers from rapid context growth, limiting scalability in larger multi-agent settings. Existing methods prune low-utility…
PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
Can Hankendi, Rana Shahout, Minlan Yu +1
Large language model (LLM) inference has become a dominant workload in modern data centers, driving significant GPU utilization and energy consumption. While prior systems optimize…
Orla: A Library for Serving LLM-Based Multi-Agent Systems
Rana Shahout, Hayder Tirmazi, Minlan Yu +1
We introduce Orla, a library for constructing and running LLM-based agentic systems. Modern agentic applications consist of workflows that combine multiple LLM inference steps, too…
Predictive Scheduling for Efficient Inference-Time Reasoning in Large Language Models
Katrina Brown, Aneesh Muppidi, Rana Shahout
Large language models (LLMs) achieve state-of-the-art accuracy on complex reasoning tasks by generating multiple chain-of-thought (CoT) traces, but using a fixed token budget per q…
Distributed Recoverable Sketches (Extended Version)
Diana Cohen, Roy Friedman, Rana Shahout
Sketches are commonly used in computer systems and network monitoring tools to provide efficient query executions while maintaining a compact data representation. Switches and rout…
From Score Distributions to Balance: Plug-and-Play Mixture-of-Experts Routing
Rana Shahout, Colin Cai, Yilun Du +2
Mixture-of-Experts (MoE) models can scale parameter capacity by routing each token to a subset of experts through a learned gate function. While conditional routing reduces trainin…
Intra-request branch orchestration for efficient LLM reasoning
Weifan Jiang, Rana Shahout, Yilun Du +2
Large Language Models (LLMs) increasingly rely on inference-time reasoning algorithms such as chain-of-thought and multi-branch reasoning to improve accuracy on complex tasks. Thes…
Queueing, Predictions, and LLMs: Challenges and Open Problems
Michael Mitzenmacher, Rana Shahout
Queueing systems present many opportunities for applying machine-learning predictions, such as estimated service times, to improve system performance. This integration raises numer…
Federated Learning Clients Clustering with Adaptation to Data Drifts
Minghao Li, Dmitrii Avdiukhin, Rana Shahout +3
Federated Learning (FL) trains deep models across edge devices without centralizing raw data, preserving user privacy. However, client heterogeneity slows down convergence and limi…
Fast Inference for Augmented Large Language Models
Rana Shahout, Cong Liang, Shiji Xin +4
Augmented Large Language Models (LLMs) enhance the capabilities of standalone LLMs by integrating external data sources through API calls. In interactive LLM applications, efficien…
Don't Stop Me Now: Embedding Based Scheduling for LLMs
Rana Shahout, Eran Malach, Chunwei Liu +3
Efficient scheduling is crucial for interactive Large Language Model (LLM) applications, where low request completion time directly impacts user engagement. Size-based scheduling a…
Learning-Augmented Frequency Estimation in Sliding Windows
Rana Shahout, Ibrahim Sabek, Michael Mitzenmacher
We show how to utilize machine learning approaches to improve sliding window algorithms for approximate frequency estimation problems, under the ``algorithms with predictions'' fra…
Learning-Based Heavy Hitters and Flow Frequency Estimation in Streams
Rana Shahout, Michael Mitzenmacher
Identifying heavy hitters and estimating the frequencies of flows are fundamental tasks in various network domains. Existing approaches to this challenge can broadly be categorized…
SkipPredict: When to Invest in Predictions for Scheduling
Rana Shahout, Michael Mitzenmacher
In light of recent work on scheduling with predicted job sizes, we consider the effect of the cost of predictions in queueing systems, removing the assumption in prior research tha…
Interface Design to Mitigate Inflation in Recommender Systems
Rana Shahout, Yehonatan Peisakhovsky, Sasha Stoikov +1
Recommendation systems rely on user-provided data to learn about item quality and provide personalized recommendations. An implicit assumption when aggregating ratings into item qu…
SQUAD: Combining Sketching and Sampling Is Better than Either for Per-item Quantile Estimation
Rana Shahout, Roy Friedman, Ran Ben Basat
Stream monitoring is fundamental in many data stream applications, such as financial data trackers, security, anomaly detection, and load balancing. In that respect, quantiles are…
Heavy Hitters over Interval Queries
Ran Ben Basat, Roy Friedman, Rana Shahout
Heavy hitters and frequency measurements are fundamental in many networking applications such as load balancing, QoS, and network security. This paper considers a generalized slidi…