3 citations · 3 across the 2 of their papers we have counts for
6 papers
Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving
Ferran Agullo, Joan Oliveras, Chen Wang +5
Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of a…
SMART-MIG: A Learning Framework for Scalable and Energy-Efficient GPU Scheduling
Wenqing Yu, Neel Karia, Tanvi Hisaria +3
The emergence of Multi-Instance GPU (MIG) technology enables us to run smaller machine learning models on partitions of a GPU rather than the entire device, thus improving utilizat…
Energy Efficient Scheduling of AI/ML Workloads on Multi Instance GPUs with Dynamic Repartitioning
Ellie Lipe, Neel Karia, Connor Espenshade +3
Increasing demand from AI/ML workloads is exacerbating the rising energy consumption of data centers. Recent advances in hardware such as NVIDIA's Multi Instance GPUs (MIGs) offer…
A Data-driven ML Approach for Maximizing Performance in LLM-Adapter Serving
Ferran Agullo, Joan Oliveras, Chen Wang +5
With the rapid adoption of Large Language Models (LLMs), LLM-adapters have become increasingly common, providing lightweight specialization of large-scale models. Serving hundreds…
Towards Pareto Optimal Throughput in Small Language Model Serving
Pol G. Recasens, Yue Zhu, Chen Wang +5
Large language models (LLMs) have revolutionized the state-of-the-art of many different natural language processing tasks. Although serving LLMs is computationally and memory deman…
Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
Pol G. Recasens, Ferran Agullo, Yue Zhu +5
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference.…