5 citations · 9 across the 15 of their papers we have counts for
15 papers
Learned JPEG Compression for DNN Vision
Kaixiang Zheng, Ahmed H. Salamah, Siyu Chen +1
JPEG, a lossy image compression technique designed for human viewers, has maintained its dominance for decades. However, in the era of artificial intelligence (AI), a substantial p…
Brain4FMs: A Benchmark of Foundation Models for Electrical Brain Signal
Fanqi Shen, Enhong Yang, Jiahe Li +5
Brain foundation models (BFMs) are advancing neurotechnology by learning transferable representations from neural signals, with broad potential in clinical diagnosis and neuroscien…
Normalized Conditional Mutual Information Surrogate Loss for Deep Neural Classifiers
Linfeng Ye, Zhixiang Chi, Konstantinos N. Plataniotis +1
In this paper, we propose a novel information theoretic surrogate loss; normalized conditional mutual information (NCMI); as a drop in alternative to the de facto cross-entropy (CE…
Coupled Data and Measurement Space Dynamics for Enhanced Diffusion Posterior Sampling
Shayan Mohajer Hamidi, En-Hui Yang, Ben Liang
Inverse problems, where the goal is to recover an unknown signal from noisy or incomplete measurements, are central to applications in medical imaging, remote sensing, and computat…
Towards Undistillable Models by Minimizing Conditional Mutual Information
Linfeng Ye, Shayan Mohajer Hamidi, En-hui Yang
A deep neural network (DNN) is said to be undistillable if, when used as a black-box input-output teacher, it cannot be distilled through knowledge distillation (KD). In this case,…
Leveraging Conditional Mutual Information to Improve Large Language Model Fine-Tuning For Classification
Thanushon Sivakaran, En-Hui Yang
Although large language models (LLMs) have demonstrated remarkable capabilities in recent years, the potential of information theory (IT) to enhance LLM development remains underex…
Coded Deep Learning: Framework and Algorithm
En-hui Yang, Shayan Mohajer Hamidi
The success of deep learning (DL) is often achieved with large models and high complexity during both training and post-training inferences, hindering training in resource-limited…
Conditional Mutual Information Based Diffusion Posterior Sampling for Solving Inverse Problems
Shayan Mohajer Hamidi, En-Hui Yang
Inverse problems are prevalent across various disciplines in science and engineering. In the field of computer vision, tasks such as inpainting, deblurring, and super-resolution ar…
Enhancing Diffusion Models for Inverse Problems with Covariance-Aware Posterior Sampling
Shayan Mohajer Hamidi, En-Hui Yang
Inverse problems exist in many disciplines of science and engineering. In computer vision, for example, tasks such as inpainting, deblurring, and super resolution can be effectivel…
Going Beyond Feature Similarity: Effective Dataset Distillation based on Class-Aware Conditional Mutual Information
Xinhao Zhong, Bin Chen, Hao Fang +3
Dataset distillation (DD) aims to minimize the time and memory consumption needed for training deep neural networks on large datasets, by creating a smaller synthetic dataset that…
JPEG Inspired Deep Learning
Ahmed H. Salamah, Kaixiang Zheng, Yiwen Liu +1
Although it is traditionally believed that lossy image compression, such as JPEG compression, has a negative impact on the performance of deep neural networks (DNNs), it is shown b…
Knowledge Distillation Based on Transformed Teacher Matching
Kaixiang Zheng, En-Hui Yang
As a technique to bridge logit matching and probability distribution matching, temperature scaling plays a pivotal role in knowledge distillation (KD). Conventionally, temperature…
AdaFed: Fair Federated Learning via Adaptive Common Descent Direction
Shayan Mohajer Hamidi, En-Hui Yang
Federated learning (FL) is a promising technology via which some edge devices/clients collaboratively train a machine learning model orchestrated by a server. Learning an unfair mo…
Bayes Conditional Distribution Estimation for Knowledge Distillation Based on Conditional Mutual Information
Linfeng Ye, Shayan Mohajer Hamidi, Renhao Tan +1
It is believed that in knowledge distillation (KD), the role of the teacher is to provide an estimate for the unknown Bayes conditional probability distribution (BCPD) to be used i…
Conditional Mutual Information Constrained Deep Learning for Classification
En-Hui Yang, Shayan Mohajer Hamidi, Linfeng Ye +2
The concepts of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) are introduced to measure the concentration and separation performance of…