1 citations · 1 across the 3 of their papers we have counts for
6 papers
Serving Heterogeneous LoRA Adapters in Distributed LLM Inference Systems
Shashwat Jaiswal, Shrikara Arun, Anjaly Parayil +8
Low-Rank Adaptation (LoRA) has become the de facto method for parameter-efficient fine-tuning of large language models (LLMs), enabling rapid adaptation to diverse domains. In prod…
COGNATE: Acceleration of Sparse Tensor Programs on Emerging Hardware using Transfer Learning
Chamika Sudusinghe, Gerasimos Gerogiannis, Damitha Lenadora +3
Sparse tensor programs are essential in deep learning and graph analytics, driving the need for optimized processing. To meet this demand, specialized hardware accelerators are bei…
DECA: A Near-Core LLM Decompression Accelerator Grounded on a 3D Roofline Model
Gerasimos Gerogiannis, Stijn Eyerman, Evangelos Georganas +2
To alleviate the memory bandwidth bottleneck in Large Language Model (LLM) inference workloads, weight matrices are stored in memory in quantized and sparsified formats. Hence, bef…
TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri +5
The rising demand for generative large language models (LLMs) poses challenges for thermal and power management in cloud datacenters. Traditional techniques often are inadequate fo…
Chameleon: Adaptive Caching and Scheduling for Many-Adapter LLM Inference Environments
Nikoleta Iliakopoulou, Jovan Stojkovic, Chloe Alverti +3
The widespread adoption of LLMs has driven an exponential rise in their deployment, imposing substantial demands on inference clusters. These clusters must handle numerous concurre…
Transforming the Hybrid Cloud for Emerging AI Workloads
Deming Chen, Alaa Youssef, Ruchi Pendse +42
This white paper, developed through close collaboration between IBM Research and UIUC researchers within the IIDAI Institute, envisions transforming hybrid cloud systems to meet th…