150 citations · 488 across the 33 of their papers we have counts for
26 papers · 1 filter
Follow the Geometry, Not the Model: Cold Start Semi-Supervised Learning
Itai David, Daphna Weinshall
Modern semi-supervised learning (SSL) couples pseudo-label generation and classifier training, using the classifier's own confidence to select the pseudo-labels that are then used…
Coordination on a Budget: Federated Active Learning with Few Labels
Liam Mohr, Daphna Weinshall
Federated Active Learning (FAL) addresses the dual challenges of data privacy and label scarcity, where the absence of a global data view introduces additional hurdles for coordina…
VISTA: Validation-Informed Trajectory Adaptation via Self-Distillation
Eli Corn, Daphna Weinshall
Deep learning models may converge to suboptimal solutions despite strong validation accuracy, masking an optimization failure we term Trajectory Deviation. This is because as train…
Leveraging Complementary Embeddings for Replay Selection in Continual Learning with Small Buffers
Danit Yanowsky, Daphna Weinshall
Catastrophic forgetting remains a key challenge in Continual Learning (CL). In replay-based CL with severe memory constraints, performance critically depends on the sample selectio…
Forget Me Not: Fighting Local Overfitting with Knowledge Fusion and Distillation
Uri Stern, Eli Corn, Daphna Weinshall
Overfitting in deep neural networks occurs less frequently than expected. This is a puzzling observation, as theory predicts that greater model capacity should eventually lead to o…
Active Learning with a Noisy Annotator
Netta Shafir, Guy Hacohen, Daphna Weinshall
Active Learning (AL) aims to reduce annotation costs by strategically selecting the most informative samples for labeling. However, most active learning methods struggle in the low…
On Local Overfitting and Forgetting in Deep Neural Networks
Uri Stern, Tomer Yaacoby, Daphna Weinshall
The infrequent occurrence of overfitting in deep neural networks is perplexing: contrary to theoretical expectations, increasing model size often enhances performance in practice.…
DCoM: Active Learning for All Learners
Inbal Mishal, Daphna Weinshall
Deep Active Learning (AL) techniques can be effective in reducing annotation costs for training deep models. However, their effectiveness in low- and high-budget scenarios seems to…
TEAL: New Selection Strategy for Small Buffers in Experience Replay Class Incremental Learning
Shahar Shaul-Ariel, Daphna Weinshall
Continual Learning is an unresolved challenge, whose relevance increases when considering modern applications. Unlike the human brain, trained deep neural networks suffer from a ph…
Relearning Forgotten Knowledge: on Forgetting, Overfit and Training-Free Ensembles of DNNs
Uri Stern, Daphna Weinshall
The infrequent occurrence of overfit in deep neural networks is perplexing. On the one hand, theory predicts that as models get larger they should eventually become too specialized…
United We Stand: Using Epoch-wise Agreement of Ensembles to Combat Overfit
Uri Stern, Daniel Shwartz, Daphna Weinshall
Deep neural networks have become the method of choice for solving many classification tasks, largely because they can fit very complex functions defined over raw data. The downside…
Pruning the Unlabeled Data to Improve Semi-Supervised Learning
Guy Hacohen, Daphna Weinshall
In the domain of semi-supervised learning (SSL), the conventional approach involves training a learner with a limited amount of labeled data alongside a substantial volume of unlab…
How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budget
Guy Hacohen, Daphna Weinshall
In the domain of Active Learning (AL), a learner actively selects which unlabeled examples to seek labels from an oracle, while operating within predefined budget constraints. Impo…
The Dynamic of Consensus in Deep Networks and the Identification of Noisy Labels
Daniel Shwartz, Uri Stern, Daphna Weinshall
Deep neural networks have incredible capacity and expressibility, and can seemingly memorize any training set. This introduces a problem when training in the presence of noisy labe…
Active Learning Through a Covering Lens
Ofer Yehuda, Avihu Dekel, Guy Hacohen +1
Deep active learning aims to reduce the annotation cost for the training of deep models, which is notoriously data-hungry. Until recently, deep active learning methods were ineffec…
Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets
Guy Hacohen, Avihu Dekel, Daphna Weinshall
Investigating active learning, we focus on the relation between the number of labeled examples (budget size), and suitable querying strategies. Our theoretical analysis shows a beh…
Principal Components Bias in Over-parameterized Linear Models, and its Manifestation in Deep Neural Networks
Guy Hacohen, Daphna Weinshall
Recent work suggests that convolutional neural networks of different architectures learn to classify images in the same order. To understand this phenomenon, we revisit the over-pa…
More Is More -- Narrowing the Generalization Gap by Adding Classification Heads
Roee Cates, Daphna Weinshall
Overfit is a fundamental problem in machine learning in general, and in deep learning in particular. In order to reduce overfit and improve generalization in the classification of…
Generative Latent Implicit Conditional Optimization when Learning from Small Sample
Idan Azuri, Daphna Weinshall
We revisit the long-standing problem of learning from a small sample, to which end we propose a novel method called GLICO (Generative Latent Implicit Conditional Optimization). GLI…
On The Power of Curriculum Learning in Training Deep Networks
Guy Hacohen, Daphna Weinshall
Training neural networks is traditionally done by providing a sequence of random mini-batches sampled uniformly from the entire training data. In this work, we analyze the effect o…
Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasets
Guy Hacohen, Leshem Choshen, Daphna Weinshall
We report a series of robust empirical observations, demonstrating that deep Neural Networks learn the examples in both the training and test sets in a similar order. This phenomen…
Theory of Curriculum Learning, with Convex Loss Functions
Daphna Weinshall, Dan Amir
Curriculum Learning - the idea of teaching by gradually exposing the learner to examples in a meaningful order, from easy to hard, has been investigated in the context of machine l…
Gaussian Mixture Generative Adversarial Networks for Diverse Datasets, and the Unsupervised Clustering of Images
Matan Ben-Yosef, Daphna Weinshall
Generative Adversarial Networks (GANs) have been shown to produce realistically looking synthetic images with remarkable success, yet their performance seems less impressive when t…
Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks
Daphna Weinshall, Gad Cohen, Dan Amir
We provide theoretical investigation of curriculum learning in the context of stochastic gradient descent when optimizing the convex linear regression loss. We prove that the rate…