3 citations · 6 across the 13 of their papers we have counts for
10 papers · 1 filter
Which Optimizer Leaks Most? Optimizer Implicit Bias as a Training-Data Privacy Controller
Sagi Habani, Yehonathan Refael, Ofir Lindenbaum
Releasing a neural network checkpoint can leak information about its training data, and existing reconstruction results rely on gradient-descent training. We ask whether the choice…
Train Less, Infer Faster: Efficient Model Finetuning and Compression via Structured Sparsity
Jonathan Svirsky, Yehonathan Refael, Ofir Lindenbaum
Fully finetuning foundation language models (LMs) with billions of parameters is often impractical due to high computational costs, memory requirements, and the risk of overfitting…
On-the-Fly OVD Adaptation with FLAME: Few-shot Localization via Active Marginal-Samples Exploration
Yehonathan Refael, Amit Aides, Aviad Barzilai +5
Open-vocabulary object detection (OVD) models offer remarkable flexibility by detecting objects from arbitrary text queries. However, their zero-shot performance in specialized dom…
SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training
Yehonathan Refael, Guy Smorodinsky, Tom Tirer +1
Low-rank gradient-based optimization methods have significantly improved memory efficiency during the training of large language models (LLMs), enabling operations within constrain…
LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM
Yehonathan Refael, Iftach Arbel, Ofir Lindenbaum +1
We study robust parameter-efficient fine-tuning (PEFT) techniques designed to improve accuracy and generalization while operating within strict computational and memory hardware co…
FineGates: LLMs Finetuning with Compression using Stochastic Gates
Jonathan Svirsky, Yehonathan Refael, Ofir Lindenbaum
Large Language Models (LLMs), with billions of parameters, present significant challenges for full finetuning due to the high computational demands, memory requirements, and imprac…
AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning
Yehonathan Refael, Jonathan Svirsky, Boris Shustin +2
Training and fine-tuning large language models (LLMs) come with challenges related to memory and computational requirements due to the increasing size of the model weights and the…
Unveiling Multiple Descents in Unsupervised Autoencoders
Kobi Rahimi, Yehonathan Refael, Tom Tirer +1
The phenomenon of double descent has challenged the traditional bias-variance trade-off in supervised learning but remains unexplored in unsupervised learning, with some studies ar…
Learning k-Level Structured Sparse Neural Networks Using Group Envelope Regularization
Yehonathan Refael, Iftach Arbel, Wasim Huleihel
The extensive need for computational resources poses a significant obstacle to deploying large-scale Deep Neural Networks (DNN) on devices with constrained resources. At the same t…
Mathematical Framework for Online Social Media Auditing
Wasim Huleihel, Yehonathan Refael
Social media platforms (SMPs) leverage algorithmic filtering (AF) as a means of selecting the content that constitutes a user's feed with the aim of maximizing their rewards. Selec…