2 citations · 4 across the 7 of their papers we have counts for
3 papers · 1 filter
MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation
Ahmad Rashid, Vasileios Lioutas, Mehdi Rezagholizadeh
The advent of large pre-trained language models has given rise to rapid progress in the field of Natural Language Processing (NLP). While the performance of these models on standar…
Towards Zero-Shot Knowledge Distillation for Natural Language Processing
Ahmad Rashid, Vasileios Lioutas, Abbas Ghaddar +1
Knowledge Distillation (KD) is a common knowledge transfer algorithm used for model compression across a variety of deep learning based natural language processing (NLP) solutions.…
Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition
Vasileios Lioutas, Ahmad Rashid, Krtin Kumar +2
Word-embeddings are vital components of Natural Language Processing (NLP) models and have been extensively explored. However, they consume a lot of memory which poses a challenge f…