2 citations · 3 across the 8 of their papers we have counts for
8 papers
Towards Undistillable Models by Minimizing Conditional Mutual Information
Linfeng Ye, Shayan Mohajer Hamidi, En-hui Yang
A deep neural network (DNN) is said to be undistillable if, when used as a black-box input-output teacher, it cannot be distilled through knowledge distillation (KD). In this case,…
JPEG Compliant Compression for Both Human and Machine, A Report
Linfeng Ye
Deep Neural Networks (DNNs) have become an integral part of our daily lives, especially in vision-related applications. However, the conventional lossy image compression algorithms…
Distributed Quasi-Newton Method for Fair and Fast Federated Learning
Shayan Mohajer Hamidi, Linfeng Ye
Federated learning (FL) is a promising technology that enables edge devices/clients to collaboratively and iteratively train a machine learning model under the coordination of a ce…
How to Train the Teacher Model for Effective Knowledge Distillation
Shayan Mohajer Hamidi, Xizhen Deng, Renhao Tan +2
Recently, it was shown that the role of the teacher in knowledge distillation (KD) is to provide the student with an estimate of the true Bayes conditional probability density (BCP…
Adversarial Training via Adaptive Knowledge Amalgamation of an Ensemble of Teachers
Shayan Mohajer Hamidi, Linfeng Ye
Adversarial training (AT) is a popular method for training robust deep neural networks (DNNs) against adversarial attacks. Yet, AT suffers from two shortcomings: (i) the robustness…
Robustness Against Adversarial Attacks via Learning Confined Adversarial Polytopes
Shayan Mohajer Hamidi, Linfeng Ye
Deep neural networks (DNNs) could be deceived by generating human-imperceptible perturbations of clean samples. Therefore, enhancing the robustness of DNNs against adversarial atta…
Bayes Conditional Distribution Estimation for Knowledge Distillation Based on Conditional Mutual Information
Linfeng Ye, Shayan Mohajer Hamidi, Renhao Tan +1
It is believed that in knowledge distillation (KD), the role of the teacher is to provide an estimate for the unknown Bayes conditional probability distribution (BCPD) to be used i…
Conditional Mutual Information Constrained Deep Learning for Classification
En-Hui Yang, Shayan Mohajer Hamidi, Linfeng Ye +2
The concepts of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) are introduced to measure the concentration and separation performance of…