73 citations · 133 across the 13 of their papers we have counts for
5 papers · 1 filter
BD-KD: Balancing the Divergences for Online Knowledge Distillation
Ibtihel Amara, Nazanin Sepahvand, Brett H. Meyer +2
We address the challenge of producing trustworthy and accurate compact models for edge devices. While Knowledge Distillation (KD) has improved model compression in terms of achievi…
CES-KD: Curriculum-based Expert Selection for Guided Knowledge Distillation
Ibtihel Amara, Maryam Ziaeefard, Brett H. Meyer +2
Knowledge distillation (KD) is an effective tool for compressing deep classification models for edge devices. However, the performance of KD is affected by the large capacity gap b…
Efficient Fine-Tuning of Compressed Language Models with Learners
Danilo Vucetic, Mohammadreza Tayaranian, Maryam Ziaeefard +3
Fine-tuning BERT-based models is resource-intensive in memory, computation, and time. While many prior works aim to improve inference efficiency via compression techniques, e.g., p…
Efficient Fine-Tuning of BERT Models on the Edge
Danilo Vucetic, Mohammadreza Tayaranian, Maryam Ziaeefard +3
Resource-constrained devices are increasingly the deployment targets of machine learning applications. Static models, however, do not always suffice for dynamic environments. On-de…
Standard Deviation-Based Quantization for Deep Neural Networks
Amir Ardakani, Arash Ardakani, Brett Meyer +2
Quantization of deep neural networks is a promising approach that reduces the inference cost, making it feasible to run deep networks on resource-restricted devices. Inspired by ex…