192 citations · 233 across the 8 of their papers we have counts for
8 papers
Privacy-Preserving Transformers: SwiftKey's Differential Privacy Implementation
Abdelrahman Abouelenin, Mohamed Abdelrehim, Raffy Fahim +2
In this paper we train a transformer using differential privacy (DP) for language modeling in SwiftKey. We run multiple experiments to balance the trade-off between the model size,…
Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
Microsoft, :, Abdelrahman Abouelenin +73
We introduce Phi-4-Mini and Phi-4-Multimodal, compact yet highly capable language and multimodal models. Phi-4-Mini is a 3.8-billion-parameter language model trained on high-qualit…
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
Amr Hendy, Mohamed Abdelrehim, Amr Sharaf +6
Generative Pre-trained Transformer (GPT) models have shown remarkable capabilities for natural language generation, but their performance for machine translation has not been thoro…
Domain Specific Sub-network for Multi-Domain Neural Machine Translation
Amr Hendy, Mohamed Abdelghaffar, Mohamed Afify +1
This paper presents Domain-Specific Sub-network (DoSS). It uses a set of masks obtained through pruning to define a sub-network for each domain and finetunes the sub-network parame…
Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation
Muhammad ElNokrashy, Amr Hendy, Mohamed Maher +2
This paper proposes a simple yet effective method to improve direct (X-to-Y) translation for both cases: zero-shot and when direct data is available. We modify the input tokens at…
Ensembling of Distilled Models from Multi-task Teachers for Constrained Resource Language Pairs
Amr Hendy, Esraa A. Gad, Mohamed Abdelghaffar +4
This paper describes our submission to the constrained track of WMT21 shared news translation task. We focus on the three relatively low resource language pairs Bengali to and from…
Scalable and Efficient MoE Training for Multitask Multilingual Models
Young Jin Kim, Ammar Ahmad Awan, Alexandre Muzio +6
The Mixture of Experts (MoE) models are an emerging class of sparsely activated deep learning models that have sublinear compute costs with respect to their parameters. In contrast…
Score Combination for Improved Parallel Corpus Filtering for Low Resource Conditions
Muhammad N. ElNokrashy, Amr Hendy, Mohamed Abdelghaffar +3
This paper describes our submission to the WMT20 sentence filtering task. We combine scores from (1) a custom LASER built for each source language, (2) a classifier built to distin…