2 papers
cs.LG2025
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh +4
LLMs are seeing growing use for applications which require large context windows, and with these large context windows KV cache activations surface as the dominant contributor to m…
cs.CL2024
TinyAgent: Function Calling at the Edge
Lutfi Eren Erdogan, Nicholas Lee, Siddharth Jha +7
Recent large language models (LLMs) have enabled the development of advanced agentic systems that can integrate various tools and APIs to fulfill user queries through function call…