21 citations · 25 across the 8 of their papers we have counts for
9 papers
Attribute Controlled Dialogue Prompting
Runcheng Liu, Ahmad Rashid, Ivan Kobyzev +2
Prompt-tuning has become an increasingly popular parameter-efficient method for adapting large pretrained language models to downstream tasks. However, both discrete prompting and…
LABO: Towards Learning Optimal Label Regularization via Bi-level Optimization
Peng Lu, Ahmad Rashid, Ivan Kobyzev +2
Regularization techniques are crucial to improving the generalization performance and training efficiency of deep neural networks. Many deep learning algorithms rely on weight deca…
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging
Peng Lu, Ivan Kobyzev, Mehdi Rezagholizadeh +3
Knowledge Distillation (KD) is a commonly used technique for improving the generalization of compact Pre-trained Language Models (PLMs) on downstream tasks. However, such methods i…
Learning Functions on Multiple Sets using Multi-Set Transformers
Kira Selby, Ahmad Rashid, Ivan Kobyzev +2
We propose a general deep architecture for learning functions on multiple permutation-invariant sets. We also show how to generalize this architecture to sets of elements of any di…
How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
Tianda Li, Ahmad Rashid, Aref Jafari +3
Knowledge Distillation (KD) is a model compression algorithm that helps transfer the knowledge of a large neural network into a smaller one. Even though KD has shown promise on a w…
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition
Abbas Ghaddar, Philippe Langlais, Ahmad Rashid +1
In this work, we examine the ability of NER models to use contextual information when predicting the type of an ambiguous entity. We introduce NRB, a new testbed carefully designed…
MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation
Ahmad Rashid, Vasileios Lioutas, Mehdi Rezagholizadeh
The advent of large pre-trained language models has given rise to rapid progress in the field of Natural Language Processing (NLP). While the performance of these models on standar…
Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition
Vasileios Lioutas, Ahmad Rashid, Krtin Kumar +2
Word-embeddings are vital components of Natural Language Processing (NLP) models and have been extensively explored. However, they consume a lot of memory which poses a challenge f…
Latent Code and Text-based Generative Adversarial Networks for Soft-text Generation
Md. Akmal Haidar, Mehdi Rezagholizadeh, Alan Do-Omri +1
Text generation with generative adversarial networks (GANs) can be divided into the text-based and code-based categories according to the type of signals used for discrimination. I…