5 papers
HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models
Yuval Domb, Hadar Sackstein, Tomer Solberg
We present HyperQuant (Hadamard, optimallY Packing, Entropy Rice-coding), a unified post-training quantization pipeline for the weights and the KV cache of large language and diffu…
Why Self-Supervised Encoders Want to Be Normal
Yuval Domb
Self-supervised learning has achieved remarkable empirical success in learning robust representations without explicit labels, most recently demonstrated within the framework of Jo…
LiteAttention: A Temporal Sparse Attention for Diffusion Transformers
Dor Shmilovich, Tony Wu, Aviad Dahan +1
Diffusion Transformers, particularly for video generation, achieve remarkable quality but suffer from quadratic attention complexity, leading to prohibitive latency. Existing accel…
The Random Coding Bound Is Tight for the Average Linear Code or Lattice
Yuval Domb, Ram Zamir, Meir Feder
In 1973, Gallager proved that the random-coding bound is exponentially tight for the random code ensemble at all rates, even below expurgation. This result explained that the rando…
Non-Random Coding Error Exponent for Lattices
Yuval Domb, Meir Feder
An upper bound on the error probability of specific lattices, based on their distance-spectrum, is constructed. The derivation is accomplished using a simple alternative to the Min…