29 citations · 46 across the 9 of their papers we have counts for
10 papers · 1 filter
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding
Abbas Ghaddar, Yimeng Wu, Sunyam Bagga +11
There is a growing body of work in recent years to develop pre-trained language models (PLMs) for the Arabic language. This work concerns addressing two major problems in existing…
JABER and SABER: Junior and Senior Arabic BERt
Abbas Ghaddar, Yimeng Wu, Ahmad Rashid +10
Language-specific pre-trained models have proven to be more accurate than multilingual ones in a monolingual evaluation setting, Arabic is no exception. However, we found that prev…
NATURE: Natural Auxiliary Text Utterances for Realistic Spoken Language Evaluation
David Alfonso-Hermelo, Ahmad Rashid, Abbas Ghaddar +2
Slot-filling and intent detection are the backbone of conversational agents such as voice assistants, and are active areas of research. Even though state-of-the-art techniques on p…
A Short Study on Compressing Decoder-Based Language Models
Tianda Li, Yassir El Mesbahi, Ivan Kobyzev +6
Pre-trained Language Models (PLMs) have been successful for a wide range of natural language processing (NLP) tasks. The state-of-the-art of PLMs, however, are extremely large to b…
Kronecker Decomposition for GPT Compression
Ali Edalati, Marzieh Tahaei, Ahmad Rashid +3
GPT is an auto-regressive Transformer-based pre-trained language model which has attracted a lot of attention in the natural language processing (NLP) domain due to its state-of-th…
Knowledge Distillation with Noisy Labels for Natural Language Understanding
Shivendra Bhardwaj, Abbas Ghaddar, Ahmad Rashid +5
Knowledge Distillation (KD) is extensively used to compress and deploy large pre-trained language models on edge devices for real-world applications. However, one neglected area of…
End-to-End Self-Debiasing Framework for Robust NLU Training
Abbas Ghaddar, Philippe Langlais, Mehdi Rezagholizadeh +1
Existing Natural Language Understanding (NLU) models have been shown to incorporate dataset biases leading to strong performance on in-distribution (ID) test sets but poor performa…
Robust Embeddings Via Distributions
Kira A. Selby, Yinong Wang, Ruizhe Wang +4
Despite recent monumental advances in the field, many Natural Language Processing (NLP) models still struggle to perform adequately on noisy domains. We propose a novel probabilist…
Towards Zero-Shot Knowledge Distillation for Natural Language Processing
Ahmad Rashid, Vasileios Lioutas, Abbas Ghaddar +1
Knowledge Distillation (KD) is a common knowledge transfer algorithm used for model compression across a variety of deep learning based natural language processing (NLP) solutions.…
From Unsupervised Machine Translation To Adversarial Text Generation
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar +2
We present a self-attention based bilingual adversarial text generator (B-GAN) which can learn to generate text from the encoder representation of an unsupervised neural machine tr…