4 citations · 12 across the 10 of their papers we have counts for
6 papers · 1 filter
Consistently Informative Soft-Label Temperature for Knowledge Distillation
Hoang-Chau Luong, Nghia Van Vo, Kaiqi Zhao +1
Knowledge distillation (KD) transfers knowledge from a high-capacity teacher to a compact student by matching their predictive distributions, with temperature scaling serving as a…
Self-Supervised Quantization-Aware Knowledge Distillation
Kaiqi Zhao, Ming Zhao
Quantization-aware training (QAT) and Knowledge Distillation (KD) are combined to achieve competitive performance in creating low-bit deep learning models. However, existing works…
Automatic Attention Pruning: Improving and Automating Model Pruning using Attentions
Kaiqi Zhao, Animesh Jain, Ming Zhao
Pruning is a promising approach to compress deep learning models in order to deploy them on resource-constrained edge devices. However, many existing pruning solutions are based on…
A Contrastive Knowledge Transfer Framework for Model Compression and Transfer Learning
Kaiqi Zhao, Yitao Chen, Ming Zhao
Knowledge Transfer (KT) achieves competitive performance and is widely used for image classification tasks in model compression and transfer learning. Existing KT works transfer th…
Enabling Deep Learning on Edge Devices through Filter Pruning and Knowledge Transfer
Kaiqi Zhao, Yitao Chen, Ming Zhao
Deep learning models have introduced various intelligent applications to edge devices, such as image classification, speech recognition, and augmented reality. There is an increasi…
Iterative Activation-based Structured Pruning
Kaiqi Zhao, Animesh Jain, Ming Zhao
Deploying complex deep learning models on edge devices is challenging because they have substantial compute and memory resource requirements, whereas edge devices' resource budget…