7 papers
Tandem Queues with Hybrid Abandonment in Heavy Traffic
Cameron Curtis, Izzy Grosof, Seyed Iravani
Hybrid-abandonment systems, where jobs abandon from certain stages of service but not others, occur frequently in medical, service, and computing systems. Abandonment affects the h…
Optimal Scheduling in Generalized Switch in Heavy Traffic
Runhan Xie, Ziv Scully, Rhonda Righter +1
The generalized switch is a highly flexible queueing model, covering multiclass, multiserver, and multiresource queueing systems as well as a wide variety of stochastic networks. A…
Multi-Turn LLM Conversations under the Least-Recently-Used Policy: Mean-Field Asymptotics and Hit Ratio Approximation
Heyuan Yao, Chutong Gao, Yuan Lyu +2
The major workloads in modern large language model (LLM) serving systems have shifted from single-shot LLM calls to multi-turn conversations, where new responses are generated base…
Pareto-Optimal Scheduling in the Half-batch Multiserver-job Model
Ziyuan Wang, Izzy Grosof
In large-scale computing systems, jobs often demand heterogeneous server allocations: large jobs that occupy a substantial fraction of the servers are of high importance and are th…
Throughput-Optimal Multiresource-Job Scheduling with Continuous Requirement Distribution
Heyuan Yao, Willow Kowalik, Izzy Grosof
Modern computing systems process jobs with resource requirements such as CPU and memory, which are described by multiresource jobs (MRJ) queueing models. In practice, job resource…
Markov Modulated JSQ in Heavy Traffic Via the Poisson Equation
Daniela Hurtado-Lange, Izzy Grosof
In parallel-server systems with a single stream of arrivals (a.k.a. load balancing), Join-the-Shortest-Queue (JSQ) is a popular routing algorithm. There is extensive literature stu…
Outperforming Multiserver SRPT at All Loads
Izzy Grosof, Daniela Hurtado-Lange
A well-designed scheduling policy can unlock significant performance improvements with no additional resources. Multiserver SRPT (SRPT-k) is known to achieve asymptotically optimal…