6 citations · 10 across the 6 of their papers we have counts for
9 papers · 1 filter
CILDA: Contrastive Data Augmentation using Intermediate Layer Knowledge Distillation
Md Akmal Haidar, Mehdi Rezagholizadeh, Abbas Ghaddar +3
Knowledge distillation (KD) is an efficient framework for compressing large-scale pre-trained language models. Recent years have seen a surge of research aiming to improve KD by le…
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation
Md Akmal Haidar, Nithin Anchuri, Mehdi Rezagholizadeh +3
Intermediate layer knowledge distillation (KD) can improve the standard KD technique (which only targets the output of teacher and student models) especially over large pre-trained…
From Unsupervised Machine Translation To Adversarial Text Generation
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar +2
We present a self-attention based bilingual adversarial text generator (B-GAN) which can learn to generate text from the encoder representation of an unsupervised neural machine tr…
A Simplified Fully Quantized Transformer for End-to-end Speech Recognition
Alex Bie, Bharat Venkitesh, Joao Monteiro +2
While significant improvements have been made in recent years in terms of end-to-end automatic speech recognition (ASR) performance, such improvements were obtained through the use…
Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition
Vasileios Lioutas, Ahmad Rashid, Krtin Kumar +2
Word-embeddings are vital components of Natural Language Processing (NLP) models and have been extensively explored. However, they consume a lot of memory which poses a challenge f…
Bilingual-GAN: A Step Towards Parallel Text Generation
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar +2
Latent space based GAN methods and attention based sequence to sequence models have achieved impressive results in text generation and unsupervised machine translation respectively…