21 citations · 25 across the 7 of their papers we have counts for
7 papers · 1 filter
Attribute Controlled Dialogue Prompting
Runcheng Liu, Ahmad Rashid, Ivan Kobyzev +2
Prompt-tuning has become an increasingly popular parameter-efficient method for adapting large pretrained language models to downstream tasks. However, both discrete prompting and…
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging
Peng Lu, Ivan Kobyzev, Mehdi Rezagholizadeh +3
Knowledge Distillation (KD) is a commonly used technique for improving the generalization of compact Pre-trained Language Models (PLMs) on downstream tasks. However, such methods i…
How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
Tianda Li, Ahmad Rashid, Aref Jafari +3
Knowledge Distillation (KD) is a model compression algorithm that helps transfer the knowledge of a large neural network into a smaller one. Even though KD has shown promise on a w…
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition
Abbas Ghaddar, Philippe Langlais, Ahmad Rashid +1
In this work, we examine the ability of NER models to use contextual information when predicting the type of an ambiguous entity. We introduce NRB, a new testbed carefully designed…
MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation
Ahmad Rashid, Vasileios Lioutas, Mehdi Rezagholizadeh
The advent of large pre-trained language models has given rise to rapid progress in the field of Natural Language Processing (NLP). While the performance of these models on standar…
Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition
Vasileios Lioutas, Ahmad Rashid, Krtin Kumar +2
Word-embeddings are vital components of Natural Language Processing (NLP) models and have been extensively explored. However, they consume a lot of memory which poses a challenge f…