5 papers
Beyond Token Savings: A Systematic Study of Context Compression in LLM Agents
Ritul Satish, Prasoon Sinha, Akiho Kawada +1
As LLM agents tackle longer tasks, they increasingly compress growing histories of reasoning, actions, and tool outputs. Compression can reduce token use, but it also changes the i…
Composer: A Search Framework for Hybrid Neural Architecture Design
Bilge Acun, Prasoon Sinha, Newsha Ardalani +7
Hybrid model architectures that combine computational primitives (e.g., Attention, MLP) in different ratios have shown promising performance beyond Transformers. Some studies have…
Oneiros: KV Cache Optimization through Parameter Remapping for Multi-tenant LLM Serving
Ruihao Li, Shagnik Pal, Vineeth Narayan Pullu +4
KV cache accelerates LLM inference by avoiding redundant computation, at the expense of memory. To support larger KV caches, prior work extends GPU memory with CPU memory via CPU-o…
iServe: An Intent-based Serving System for LLMs
Dimitrios Liakopoulos, Tianrui Hu, Prasoon Sinha +1
Large Language Models (LLMs) are becoming ubiquitous across industries, where applications demand they fulfill diverse user intents. However, developers currently face the challeng…
Shabari: Delayed Decision-Making for Faster and Efficient Serverless Functions
Prasoon Sinha, Kostis Kaffes, Neeraja J. Yadwadkar
Serverless computing relieves developers from the burden of resource management, thus providing ease-of-use to the users and the opportunity to optimize resource utilization for th…