Showing 2024Show all
2 papers · 1 filter
cs.LG2024
CompAct: Compressed Activations for Memory-Efficient LLM Training
Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki +1
We introduce CompAct, a technique that reduces peak memory utilization on GPU by 25-30% for pretraining and 50% for fine-tuning of LLMs. Peak device memory is a major limiting fact…
cs.LG2024
DNCs Require More Planning Steps
Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki +1
Many recent works use machine learning models to solve various complex algorithmic problems. However, these models attempt to reach a solution without considering the problem's req…