10 papers
Simulate Before You Scale: High-Fidelity LLM Inference Planning Without GPUs
Dipanwita Guhathakurta, Mert Toslali, Srinivasan Parthasarathy +6
Deploying large language model (LLM) inference at scale requires jointly selecting hardware layout, parallelism strategy, scheduling policy, admission control, and request routing…
ROAR: Unifying Runs across Heterogeneous AI-Driven Research Systems
Leo Y. Lin, Vishakha Ramani, Z. Berkay Celik +2
Each run of an AI-driven research system (ADRS) is an expensive search over a vast solution space, and dependable evaluation requires many runs, making run data both costly to prod…
An Approximate Queueing Model of LLM Inference Serving for SLO-Driven Autoscaling
Vishakha Ramani, Asser N. Tantawi
Performance models of LLM servers support both latency evaluation and the design of controllers for autoscaling against service level objectives (SLOs) and for inference optimizati…
WVA: A Global Optimization Control Plane for llmd
Abhishek Malvankar, Lionel Villard, Mohammed Abdi +6
As Large Language Models (LLMs) scale to handle massive concurrent traffic, optimizing the infrastructure required for inference has become a primary challenge. To manage the high…
Timely and Energy-Efficient Multi-Step Update Processing
Vishakha Ramani, Ivan Seskar, Roy D. Yates
This work explores systems where source updates require multiple sequential processing steps. We model and analyze the Age of Information (AoI) performance of various system design…
Efficient and Timely Memory Access
Vishakha Ramani, Ivan Seskar, Roy D. Yates
This paper investigates the optimization of memory sampling in status updating systems, where source updates are published in shared memory, and reader process samples the memory f…
Age-Memory Trade-off in Read-Copy-Update
Vishakha Ramani, Jiachen Chen, Roy D. Yates
In the realm of shared memory systems, the challenge of reader-writer synchronization is closely coupled with the potential for readers to access outdated updates. Read-Copy-Update…
Timely Processing Of Updates From Multiple Sources
Vishakha Ramani, Ivan Seskar, Roy D. Yates
We consider a system where the updates from independent sources are disseminated via a publish-subscribe mechanism. The sources are the publishers and a decision process (DP), acti…
Timely Mobile Routing: An Experimental Study
Vishakha Ramani, Jiachen Chen, Roy D. Yates
Time-critical cyber-physical applications demand the timely delivery of information. In this work, we employ a high-speed packet processing testbed to quantitatively analyze a pack…
Lock-based or Lock-less: Which Is Fresh?
Vishakha Ramani, Jiachen Chen, Roy D. Yates
We examine status updating systems in which time-stamped status updates are stored/written in shared-memory. Specifically, we compare Read-Copy-Update (RCU) and Readers-Writer lock…