5 papers
How Does Overparameterization Affect Machine Unlearning of Deep Neural Networks?
Gal Alon, Yehuda Dar
Machine unlearning is the task of updating a trained model to forget specific training data without retraining from scratch. In this paper, we investigate how unlearning of deep ne…
Transfer Learning of Linear Regression with Multiple Pretrained Models: Benefiting from More Pretrained Models via Overparameterization Debiasing
Daniel Boharon, Yehuda Dar
We study transfer learning for a linear regression task using several least-squares pretrained models that can be overparameterized. We formulate the target learning task as optimi…
Mixture of Many Zero-Compute Experts: A High-Rate Quantization Theory Perspective
Yehuda Dar
This paper uses classical high-rate quantization theory to provide new insights into mixture-of-experts (MoE) models for regression tasks. Our MoE is defined by a segmentation of t…
How Do the Architecture and Optimizer Affect Representation Learning? On the Training Dynamics of Representations in Deep Neural Networks
Yuval Sharon, Yehuda Dar
In this paper, we elucidate how representations in deep neural networks (DNNs) evolve during training. Our focus is on overparameterized learning settings where the training contin…
TL-PCA: Transfer Learning of Principal Component Analysis
Sharon Hendy, Yehuda Dar
Principal component analysis (PCA) can be significantly limited when there is too few examples of the target data of interest. We propose a transfer learning approach to PCA (TL-PC…