6 papers
AnchorKV: Anchor-Residual KV Cache Compression
Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2
The key-value (KV) cache is the primary memory bottleneck in long-context LLM inference. Existing approaches attack it from opposite ends: eviction methods permanently discard toke…
MACRO: Training-free Multi-plane Attention for Closeup Render Optimization
Nitzan Hodos, Roy Amoyal, Lior Fritz +3
Close-up rendering, zooming into a scene well beyond any training camera, is important for virtual production and interactive 3D content, yet remains an open challenge. 3D Gaussian…
QKV Projections Require a Fraction of Their Memory
Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2
The Multi-Head Attention mechanism is central to LLM operation, and multiple works target its compute and memory efficiency during training. While most works focus on approximating…
CompAct: Compressed Activations for Memory-Efficient LLM Training
Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki +1
We introduce CompAct, a technique that reduces peak memory utilization on GPU by 25-30% for pretraining and 50% for fine-tuning of LLMs. Peak device memory is a major limiting fact…
DNCs Require More Planning Steps
Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki +1
Many recent works use machine learning models to solve various complex algorithmic problems. However, these models attempt to reach a solution without considering the problem's req…
Coin Flipping Neural Networks
Yuval Sieradzki, Nitzan Hodos, Gal Yehuda +1
We show that neural networks with access to randomness can outperform deterministic networks by using amplification. We call such networks Coin-Flipping Neural Networks, or CFNNs.…