4 papers
HoliBench: A Cross-Platform Benchmarking and Deployment Toolkit for Foundation Models in CPS-IoT Applications
Inesh Chakrabarti, Zejun Xiong, Pragya Sharma +1
Foundation models, including large language models, vision-language models, and time-series foundation models, are increasingly deployed on embedded and edge platforms for CPS and…
Shape Mutating Expert Compression:LorExperts and BTExperts
Inesh Chakrabarti, Sourjya Roy, Bowen Bao +3
Mixture-of-Experts (MoE) language models deliver high capacity at low per-token compute, but deploying them cheaply requires compressing their many expert weight matrices. Expert p…
UltraQuant: 4-bit KV Caching for Context-Heavy Agents
Inesh Chakrabarti, David Limpus, Aditi Ghai Rana +4
Context-heavy agents place substantial pressure on the key-value (KV) cache: long prefixes are reused across many short turns, while concurrency determines whether the serving syst…
NoWag: A Unified Framework for Shape Preserving Compression of Large Language Models
Lawrence Liu, Inesh Chakrabarti, Yixiao Li +3
Large language models (LLMs) exhibit remarkable performance across various natural language processing tasks but suffer from immense computational and memory demands, limiting thei…