704 citations · 3.6k across the 98 of their papers we have counts for
25 papers · 1 filter
An information-Theoretic Approach to Semi-supervised Transfer Learning
Daniel Jakubovitz, David Uliel, Miguel Rodrigues +1
Transfer learning is a valuable tool in deep learning as it allows propagating information from one "source dataset" to another "target dataset", especially in the case of a small…
Pruning at Initialization -- A Sketching Perspective
Noga Bar, Raja Giryes
The lottery ticket hypothesis (LTH) has increased attention to pruning neural networks at initialization. We study this problem in the linear setting. We show that finding a sparse…
Neural (Tangent Kernel) Collapse
Mariia Seleznova, Dana Weitzner, Raja Giryes +2
This work bridges two important concepts: the Neural Tangent Kernel (NTK), which captures the evolution of deep neural networks (DNNs) during training, and the Neural Collapse (NC)…
Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets
Edo Cohen-Karlik, Itamar Menuhin-Gruman, Raja Giryes +2
Overparameterization in deep learning typically refers to settings where a trained neural network (NN) has representational capacity to fit the training data in many ways, some of…
Utilizing Excess Resources in Training Neural Networks
Amit Henig, Raja Giryes
In this work, we suggest Kernel Filtering Linear Overparameterization (KFLO), where a linear cascade of filtering layers is used during training to improve network performance in t…
Membership Inference Attack Using Self Influence Functions
Gilad Cohen, Raja Giryes
Member inference (MI) attacks aim to determine if a specific data sample was used to train a machine learning model. Thus, MI is a major privacy threat to models trained on private…
SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimization
Amir Hertz, Or Perel, Raja Giryes +2
Multilayer-perceptrons (MLP) are known to struggle with learning functions of high-frequencies, and in particular cases with wide frequency bands. We present a spatially adaptive p…
Multiplicative Reweighting for Robust Neural Network Optimization
Noga Bar, Tomer Koren, Raja Giryes
Neural networks are widespread due to their powerful performance. Yet, they degrade in the presence of noisy labels at training time. Inspired by the setting of learning with exper…
GMM-Based Generative Adversarial Encoder Learning
Yuri Feigin, Hedva Spitzer, Raja Giryes
While GAN is a powerful model for generating images, its inability to infer a latent space directly limits its use in applications requiring an encoder. Our paper presents a simple…
Kernel-Based Smoothness Analysis of Residual Networks
Tom Tirer, Joan Bruna, Raja Giryes
A major factor in the success of deep neural networks is the use of sophisticated architectures rather than the classical multilayer perceptron (MLP). Residual networks (ResNets) s…
Self-supervised Neural Architecture Search
Sapir Kaplan, Raja Giryes
Neural Architecture Search (NAS) has been used recently to achieve improved performance in various tasks and most prominently in image classification. Yet, current search strategie…
A function space analysis of finite neural networks with insights from sampling theory
Raja Giryes
This work suggests using sampling theory to analyze the function space represented by neural networks. First, it shows, under the assumption of a finite input domain, which is the…
PointGMM: a Neural GMM Network for Point Clouds
Amir Hertz, Rana Hanocka, Raja Giryes +1
Point clouds are a popular representation for 3D shapes. However, they encode a particular sampling without accounting for shape priors or non-local information. We advocate for th…
Autoencoders
Dor Bank, Noam Koenigstein, Raja Giryes
An autoencoder is a specific type of a neural network, which is mainly designed to encode the input into a compressed and meaningful representation, and then decode it back such th…
Detecting Adversarial Samples Using Influence Functions and Nearest Neighbors
Gilad Cohen, Guillermo Sapiro, Raja Giryes
Deep neural networks (DNNs) are notorious for their vulnerability to adversarial attacks, which are small perturbations added to their input images to mislead their prediction. Det…
Lautum Regularization for Semi-supervised Transfer Learning
Daniel Jakubovitz, Miguel R. D. Rodrigues, Raja Giryes
Transfer learning is a very important tool in deep learning as it allows propagating information from one "source dataset" to another "target dataset", especially in the case of a…
Efficient non-uniform quantizer for quantized neural network targeting reconfigurable hardware
Natan Liss, Chaim Baskin, Avi Mendelson +2
Convolutional Neural Networks (CNN) has become more popular choice for various tasks such as computer vision, speech recognition and natural language processing. Thanks to their la…
An ETF view of Dropout regularization
Dor Bank, Raja Giryes
Dropout is a popular regularization technique in deep learning. Yet, the reason for its success is still not fully understood. This paper provides a new interpretation of Dropout f…
MeshCNN: A Network with an Edge
Rana Hanocka, Amir Hertz, Noa Fish +3
Polygonal meshes provide an efficient representation for 3D shapes. They explicitly capture both shape surface and topology, and leverage non-uniformity to represent large flat reg…
Generalization Error in Deep Learning
Daniel Jakubovitz, Raja Giryes, Miguel R. D. Rodrigues
Deep learning models have lately shown great performance in various fields such as computer vision, speech recognition, speech translation, and natural language processing. However…
DNN or k-NN: That is the Generalize vs. Memorize Question
Gilad Cohen, Guillermo Sapiro, Raja Giryes
This paper studies the relationship between the classification performed by deep neural networks (DNNs) and the decision of various classical classifiers, namely k-nearest neighbou…
UNIQ: Uniform Noise Injection for Non-Uniform Quantization of Neural Networks
Chaim Baskin, Eli Schwartz, Evgenii Zheltonozhskii +4
We present a novel method for neural network quantization that emulates a non-uniform -quantile quantizer, which adapts to the distribution of the quantized parameters. Our appr…
The Resistance to Label Noise in K-NN and DNN Depends on its Concentration
Amnon Drory, Oria Ratzon, Shai Avidan +1
We investigate the classification performance of K-nearest neighbors (K-NN) and deep neural networks (DNNs) in the presence of label noise. We first show empirically that a DNN's p…
Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization
Daniel Jakubovitz, Raja Giryes
Deep neural networks have lately shown tremendous performance in various applications including vision and speech processing tasks. However, alongside their ability to perform thes…