18 citations · 44 across the 10 of their papers we have counts for
12 papers
Conformer with dual-mode chunked attention for joint online and offline ASR
Felix Weninger, Marco Gaudesi, Md Akmal Haidar +3
In this paper, we present an in-depth study on online attention mechanisms and distillation techniques for dual-mode (i.e., joint online and offline) ASR using the Conformer Transd…
CILDA: Contrastive Data Augmentation using Intermediate Layer Knowledge Distillation
Md Akmal Haidar, Mehdi Rezagholizadeh, Abbas Ghaddar +3
Knowledge distillation (KD) is an efficient framework for compressing large-scale pre-trained language models. Recent years have seen a surge of research aiming to improve KD by le…
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation
Md Akmal Haidar, Nithin Anchuri, Mehdi Rezagholizadeh +3
Intermediate layer knowledge distillation (KD) can improve the standard KD technique (which only targets the output of teacher and student models) especially over large pre-trained…
Transformer-based ASR Incorporating Time-reduction Layer and Fine-tuning with Self-Knowledge Distillation
Md Akmal Haidar, Chao Xing, Mehdi Rezagholizadeh
End-to-end automatic speech recognition (ASR), unlike conventional ASR, does not have modules to learn the semantic representation from speech encoder. Moreover, the higher frame-r…
Fine-tuning of Pre-trained End-to-end Speech Recognition with Generative Adversarial Networks
Md Akmal Haidar, Mehdi Rezagholizadeh
Adversarial training of end-to-end (E2E) ASR systems using generative adversarial networks (GAN) has recently been explored for low-resource ASR corpora. GANs help to learn the tru…
From Unsupervised Machine Translation To Adversarial Text Generation
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar +2
We present a self-attention based bilingual adversarial text generator (B-GAN) which can learn to generate text from the encoder representation of an unsupervised neural machine tr…
A Simplified Fully Quantized Transformer for End-to-end Speech Recognition
Alex Bie, Bharat Venkitesh, Joao Monteiro +2
While significant improvements have been made in recent years in terms of end-to-end automatic speech recognition (ASR) performance, such improvements were obtained through the use…
Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition
Vasileios Lioutas, Ahmad Rashid, Krtin Kumar +2
Word-embeddings are vital components of Natural Language Processing (NLP) models and have been extensively explored. However, they consume a lot of memory which poses a challenge f…
Bilingual-GAN: A Step Towards Parallel Text Generation
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar +2
Latent space based GAN methods and attention based sequence to sequence models have achieved impressive results in text generation and unsupervised machine translation respectively…
TextKD-GAN: Text Generation using KnowledgeDistillation and Generative Adversarial Networks
Md. Akmal Haidar, Mehdi Rezagholizadeh
Text generation is of particular interest in many NLP applications such as machine translation, language modeling, and text summarization. Generative adversarial networks (GANs) ac…
Latent Code and Text-based Generative Adversarial Networks for Soft-text Generation
Md. Akmal Haidar, Mehdi Rezagholizadeh, Alan Do-Omri +1
Text generation with generative adversarial networks (GANs) can be divided into the text-based and code-based categories according to the type of signals used for discrimination. I…
SALSA-TEXT : self attentive latent space based adversarial text generation
Jules Gagnon-Marchand, Hamed Sadeghi, Md. Akmal Haidar +1
Inspired by the success of self attention mechanism and Transformer architecture in sequence transduction and image generation applications, we propose novel self attention-based a…