125 citations · 126 across the 3 of their papers we have counts for
4 papers · 1 filter
PLUMAGE: Probabilistic Low rank Unbiased Min Variance Gradient Estimator for Efficient Large Model Training
Matan Haroush, Daniel Soudry
Accelerator memory and networking constraints have emerged as dominant bottlenecks when training large language models LLMs with billions of parameters. Existing low rank gradient…
A statistical framework for efficient out of distribution detection in deep neural networks
Matan Haroush, Tzviel Frostig, Ruth Heller +1
Background. Commonly, Deep Neural Networks (DNNs) generalize well on samples drawn from a distribution similar to that of the training set. However, DNNs' predictions are brittle a…
The Knowledge Within: Methods for Data-Free Model Compression
Matan Haroush, Itay Hubara, Elad Hoffer +1
Recently, an extensive amount of research has been focused on compressing and accelerating Deep Neural Networks (DNN). So far, high compression rate algorithms require part of the…
Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning
Tom Zahavy, Matan Haroush, Nadav Merlis +2
Learning how to act when there are many available actions in each state is a challenging task for Reinforcement Learning (RL) agents, especially when many of the actions are redund…