57 citations · 119 across the 20 of their papers we have counts for
24 papers
Transfer Learning of Linear Regression with Multiple Pretrained Models: Benefiting from More Pretrained Models via Overparameterization Debiasing
Daniel Boharon, Yehuda Dar
We study transfer learning for a linear regression task using several least-squares pretrained models that can be overparameterized. We formulate the target learning task as optimi…
Mixture of Many Zero-Compute Experts: A High-Rate Quantization Theory Perspective
Yehuda Dar
This paper uses classical high-rate quantization theory to provide new insights into mixture-of-experts (MoE) models for regression tasks. Our MoE is defined by a segmentation of t…
How Does Overparameterization Affect Machine Unlearning of Deep Neural Networks?
Gal Alon, Yehuda Dar
Machine unlearning is the task of updating a trained model to forget specific training data without retraining from scratch. In this paper, we investigate how unlearning of deep ne…
TL-PCA: Transfer Learning of Principal Component Analysis
Sharon Hendy, Yehuda Dar
Principal component analysis (PCA) can be significantly limited when there is too few examples of the target data of interest. We propose a transfer learning approach to PCA (TL-PC…
How Do the Architecture and Optimizer Affect Representation Learning? On the Training Dynamics of Representations in Deep Neural Networks
Yuval Sharon, Yehuda Dar
In this paper, we elucidate how representations in deep neural networks (DNNs) evolve during training. Our focus is on overparameterized learning settings where the training contin…
How Much Training Data is Memorized in Overparameterized Autoencoders? An Inverse Problem Perspective on Memorization Evaluation
Koren Abitbul, Yehuda Dar
Overparameterized autoencoder models often memorize their training data. For image data, memorization is often examined by using the trained autoencoder to recover missing regions…
Frozen Overparameterization: A Double Descent Perspective on Transfer Learning of Deep Neural Networks
Yehuda Dar, Lorenzo Luzi, Richard G. Baraniuk
We study the generalization behavior of transfer learning of deep neural networks (DNNs). We adopt the overparameterization perspective -- featuring interpolation of the training d…
Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspective
Gowthami Somepalli, Liam Fowl, Arpit Bansal +5
We discuss methods for visualizing neural network decision boundaries and decision regions. We use these visualizations to investigate issues related to reproducibility and general…
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning
Yehuda Dar, Vidya Muthukumar, Richard G. Baraniuk
The last decade of progress in machine learning (ML), especially the deep learning era, has raised a number of scientific questions that challenge the longstanding dogma of the fie…
Double Descent and Other Interpolation Phenomena in GANs
Lorenzo Luzi, Yehuda Dar, Richard Baraniuk
We study overparameterization in generative adversarial networks (GANs) that can interpolate the training data. We show that overparameterization can improve generalization perform…
The Common Intuition to Transfer Learning Can Win or Lose: Case Studies for Linear Regression
Yehuda Dar, Daniel LeJeune, Richard G. Baraniuk
We study a fundamental transfer learning process from source to target linear regression tasks, including overparameterized settings where there are more learned parameters than da…
Regularized Compression of MRI Data: Modular Optimization of Joint Reconstruction and Coding
Veronica Corona, Yehuda Dar, Guy Williams +1
The Magnetic Resonance Imaging (MRI) processing chain starts with a critical acquisition stage that provides raw data for reconstruction of images for medical diagnosis. This flow…
Double Double Descent: On Generalization Errors in Transfer Learning between Linear Regression Tasks
Yehuda Dar, Richard G. Baraniuk
We study the transfer learning process between two linear regression problems. An important and timely special case is when the regressors are overparameterized and perfectly inter…
Subspace Fitting Meets Regression: The Effects of Supervision and Orthonormality Constraints on Double Descent of Generalization Errors
Yehuda Dar, Paul Mayer, Lorenzo Luzi +1
We study the linear subspace fitting problem in the overparameterized setting, where the estimated subspace can perfectly interpolate the training examples. Our scope includes the…
Algorithms for Piecewise Constant Signal Approximations
Leif Bergerhoff, Joachim Weickert, Yehuda Dar
We consider the problem of finding optimal piecewise constant approximations of one-dimensional signals. These approximations should consist of a specified number of segments (samp…
Benefiting from Duplicates of Compressed Data: Shift-Based Holographic Compression of Images
Yehuda Dar, Alfred M. Bruckstein
Storage systems often rely on multiple copies of the same compressed data, enabling recovery in case of binary data errors, of course, at the expense of a higher storage cost. In t…
Compression for Multiple Reconstructions
Yehuda Dar, Michael Elad, Alfred M. Bruckstein
In this work we propose a method for optimizing the lossy compression for a network of diverse reconstruction systems. We focus on adapting a standard image compression method to a…
System-Aware Compression
Yehuda Dar, Michael Elad, Alfred M. Bruckstein
Many information systems employ lossy compression as a crucial intermediate stage among other processing components. While the important distortion is defined by the system's input…
Optimized Pre-Compensating Compression
Yehuda Dar, Michael Elad, Alfred M. Bruckstein
In imaging systems, following acquisition, an image/video is transmitted or stored and eventually presented to human observers using different and often imperfect display devices.…
Restoration by Compression
Yehuda Dar, Michael Elad, Alfred M. Bruckstein
In this paper we study the topic of signal restoration using complexity regularization, quantifying the compression bit-cost of the signal estimate. While complexity-regularized re…
On High-Resolution Adaptive Sampling of Deterministic Signals
Yehuda Dar, Alfred M. Bruckstein
In this work we study the topic of high-resolution adaptive sampling of a given deterministic signal and establish a connection with classic approaches to high-rate quantization. S…
Postprocessing of Compressed Images via Sequential Denoising
Yehuda Dar, Alfred M. Bruckstein, Michael Elad +1
In this work we propose a novel postprocessing technique for compression-artifact reduction. Our approach is based on posing this task as an inverse problem, with a regularization…
Motion-Compensated Coding and Frame-Rate Up-Conversion: Models and Analysis
Yehuda Dar, Alfred M. Bruckstein
Block-based motion estimation (ME) and compensation (MC) techniques are widely used in modern video processing algorithms and compression systems. The great variety of video applic…
Improving Low Bit-Rate Video Coding using Spatio-Temporal Down-Scaling
Yehuda Dar, Alfred M. Bruckstein
Good quality video coding for low bit-rate applications is important for transmission over narrow-bandwidth channels and for storage with limited memory capacity. In this work, we…