5 papers
KVComm: Enabling Efficient LLM Communication through Selective KV Sharing
Xiangyu Shi, Marco Chiesa, Gerald Q. Maguire +1
Large Language Models (LLMs) are increasingly deployed in multi-agent systems, where effective inter-model communication is crucial. Existing communication protocols either rely on…
CASTILLO: Characterizing Response Length Distributions of Large Language Models
Daniel F. Perez-Ramirez, Dejan Kostic, Magnus Boman
Efficiently managing compute resources for Large Language Model (LLM) inference remains challenging due to the inherently stochastic and variable lengths of autoregressive text gen…
Priority-Aware Preemptive Scheduling for Mixed-Priority Workloads in MoE Inference
Mohammad Siavashi, Faezeh Keshmiri Dindarloo, Dejan Kostic +1
Large Language Models have revolutionized natural language processing, yet serving them efficiently in data centers remains challenging due to mixed workloads comprising latency-se…
Automating the Detection of Code Vulnerabilities by Analyzing GitHub Issues
Daniele Cipollone, Changjie Wang, Mariano Scazzariello +4
In today's digital landscape, the importance of timely and accurate vulnerability detection has significantly increased. This paper presents a novel approach that leverages transfo…
Deriving Coding-Specific Sub-Models from LLMs using Resource-Efficient Pruning
Laura Puccioni, Alireza Farshin, Mariano Scazzariello +3
Large Language Models (LLMs) have demonstrated their exceptional performance in various complex code generation tasks. However, their broader adoption is limited by significant com…