6 papers
AnchorKV: Anchor-Residual KV Cache Compression
Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2
The key-value (KV) cache is the primary memory bottleneck in long-context LLM inference. Existing approaches attack it from opposite ends: eviction methods permanently discard toke…
DiT-Flow: Speech Enhancement Robust to Multiple Distortions based on Flow Matching in Latent Space and Diffusion Transformers
Tianyu Cao, Helin Wang, Ari Frummer +7
Recent advances in generative models, such as diffusion and flow matching, have shown strong performance in audio tasks. However, speech enhancement (SE) models are typically train…
QKV Projections Require a Fraction of Their Memory
Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2
The Multi-Head Attention mechanism is central to LLM operation, and multiple works target its compute and memory efficiency during training. While most works focus on approximating…
ReFESS-QI: Reference-Free Evaluation For Speech Separation With Joint Quality And Intelligibility Scoring
Ari Frummer, Helin Wang, Tianyu Cao +6
Source separation is a crucial pre-processing step for various speech processing tasks, such as automatic speech recognition (ASR). Traditionally, the evaluation metrics for speech…
CompAct: Compressed Activations for Memory-Efficient LLM Training
Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki +1
We introduce CompAct, a technique that reduces peak memory utilization on GPU by 25-30% for pretraining and 50% for fine-tuning of LLMs. Peak device memory is a major limiting fact…
DNCs Require More Planning Steps
Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki +1
Many recent works use machine learning models to solve various complex algorithmic problems. However, these models attempt to reach a solution without considering the problem's req…