9 citations · 12 across the 18 of their papers we have counts for
5 papers · 1 filter
Squeezed Attention: Accelerating Long Context Length LLM Inference
Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh +6
Emerging Large Language Model (LLM) applications require long input context in order to perform complex tasks like document analysis and code generation. For these long context len…
Towards Scalable and Versatile Weight Space Learning
Konstantin Schürholt, Michael W. Mahoney, Damian Borth
Learning representations of well-trained neural network models holds the promise to provide an understanding of the inner workings of those models. However, previous work has eithe…
AI and Memory Wall
Amir Gholami, Zhewei Yao, Sehoon Kim +3
The availability of unprecedented unsupervised training data, along with neural scaling laws, has resulted in an unprecedented surge in model size and compute requirements for serv…
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement
Nicholas Lee, Thanakul Wattanawong, Sehoon Kim +6
Pretrained large language models (LLMs) are currently state-of-the-art for solving the vast majority of natural language processing tasks. While many real-world applications still…
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh +4
LLMs are seeing growing use for applications which require large context windows, and with these large context windows KV cache activations surface as the dominant contributor to m…