4 citations · 8 across the 8 of their papers we have counts for
13 papers
ECO: Quantized Training without Full-Precision Master Weights
Mahdi Nikdan, Amir Zandieh, Dan Alistarh +1
Quantization has significantly improved the compute and memory efficiency of Large Language Model (LLM) training. However, existing approaches still rely on accumulating their upda…
TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
Amir Zandieh, Majid Daliri, Majid Hadian +1
Vector quantization, a problem rooted in Shannon's source coding theory, aims to quantize high-dimensional Euclidean vectors while minimizing distortion in their geometric structur…
PolarQuant: Quantizing KV Caches with Polar Transformation
Insu Han, Praneeth Kacham, Amin Karbasi +2
Large language models (LLMs) require significant memory to store Key-Value (KV) embeddings in their KV cache, especially when handling long-range contexts. Quantization of these KV…
Streaming Attention Approximation via Discrepancy Theory
Ekaterina Kochetkova, Kshiteej Sheth, Insu Han +2
Large language models (LLMs) have achieved impressive success, but their high memory requirements present challenges for long-context token generation. In this paper we study the s…
Fast Neural Kernel Embeddings for General Activations
Insu Han, Amir Zandieh, Jaehoon Lee +3
Infinite width limit has shed light on generalization and optimization aspects of deep learning by establishing connections between neural networks and kernel methods. Despite thei…
Near Optimal Reconstruction of Spherical Harmonic Expansions
Amir Zandieh, Insu Han, Haim Avron
We propose an algorithm for robust recovery of the spherical harmonic expansion of functions defined on the d-dimensional unit sphere using a near-optimal number…