10 citations · 13 across the 5 of their papers we have counts for
6 papers
SMART-MIG: A Learning Framework for Scalable and Energy-Efficient GPU Scheduling
Wenqing Yu, Neel Karia, Tanvi Hisaria +3
The emergence of Multi-Instance GPU (MIG) technology enables us to run smaller machine learning models on partitions of a GPU rather than the entire device, thus improving utilizat…
Energy Efficient Scheduling of AI/ML Workloads on Multi Instance GPUs with Dynamic Repartitioning
Ellie Lipe, Neel Karia, Connor Espenshade +3
Increasing demand from AI/ML workloads is exacerbating the rising energy consumption of data centers. Recent advances in hardware such as NVIDIA's Multi Instance GPUs (MIGs) offer…
Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving
Ferran Agullo, Joan Oliveras, Chen Wang +5
Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of a…
A Data-driven ML Approach for Maximizing Performance in LLM-Adapter Serving
Ferran Agullo, Joan Oliveras, Chen Wang +5
With the rapid adoption of Large Language Models (LLMs), LLM-adapters have become increasingly common, providing lightweight specialization of large-scale models. Serving hundreds…
Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
Pol G. Recasens, Ferran Agullo, Yue Zhu +5
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference.…
Towards Pareto Optimal Throughput in Small Language Model Serving
Pol G. Recasens, Yue Zhu, Chen Wang +5
Large language models (LLMs) have revolutionized the state-of-the-art of many different natural language processing tasks. Although serving LLMs is computationally and memory deman…