4 papers
Multi-Turn LLM Conversations under the Least-Recently-Used Policy: Mean-Field Asymptotics and Hit Ratio Approximation
Heyuan Yao, Chutong Gao, Yuan Lyu +2
The major workloads in modern large language model (LLM) serving systems have shifted from single-shot LLM calls to multi-turn conversations, where new responses are generated base…
Throughput-Optimal Multiresource-Job Scheduling with Continuous Requirement Distribution
Heyuan Yao, Willow Kowalik, Izzy Grosof
Modern computing systems process jobs with resource requirements such as CPU and memory, which are described by multiresource jobs (MRJ) queueing models. In practice, job resource…
Sharp Decoupling Inequalities for the Variances and Second Moments of Sums of Dependent Random Variables
Victor H. de la Pena, Heyuan Yao, Demissie Alemayehu
Both complete decoupling and tangent decoupling are classical tools aiming to compare two random processes where one has a weaker dependence structure. We give a new proof for the…
A Scalable Formula for the Moments of a Family of Self-Normalized Statistics
Haolin Zou, Heyuan Yao, Victor de la Peña
Following the student t-statistic, normalization has been a widely used method in statistic and other disciplines including economics, ecology and machine learning. We focus on sta…