6 papers
Quantile-Guided Diffusion for Probabilistic Time-Series Forecasting
Shadi Omari, Bashar Khoury, Assaf Schuster
Quantile forecasts summarize uncertainty at each future time step, while diffusion forecasters represent it through sampled future trajectories. We study how predicted quantiles ca…
AnchorKV: Anchor-Residual KV Cache Compression
Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2
The key-value (KV) cache is the primary memory bottleneck in long-context LLM inference. Existing approaches attack it from opposite ends: eviction methods permanently discard toke…
QKV Projections Require a Fraction of Their Memory
Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2
The Multi-Head Attention mechanism is central to LLM operation, and multiple works target its compute and memory efficiency during training. While most works focus on approximating…
Large Language Models for Water Distribution Systems Modeling and Decision-Making
Yinon Goldshtein, Gal Perelman, Assaf Schuster +1
The integration of Large Language Models (LLMs) into engineering workflows presents new opportunities for making computational tools more accessible. Especially where such tools re…
CompAct: Compressed Activations for Memory-Efficient LLM Training
Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki +1
We introduce CompAct, a technique that reduces peak memory utilization on GPU by 25-30% for pretraining and 50% for fine-tuning of LLMs. Peak device memory is a major limiting fact…
DNCs Require More Planning Steps
Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki +1
Many recent works use machine learning models to solve various complex algorithmic problems. However, these models attempt to reach a solution without considering the problem's req…