A Survey of Large Language Models for Arabic Language and its Dialects
arXiv:2410.20238 · doi:10.1145/3807946
Abstract
This survey offers a comprehensive overview of Large Language Models (LLMs) designed for Arabic language and its dialects. It covers key architectures, including encoder-only, decoder-only, and encoder-decoder models, along with the datasets used for pre-training, spanning Classical Arabic, Modern Standard Arabic, and Dialectal Arabic. The study also explores monolingual, bilingual, and multilingual LLMs, analyzing their architectures and performance across downstream tasks, such as sentiment analysis, named entity recognition, and question answering. Furthermore, it assesses the openness of Arabic LLMs based on factors, such as source code availability, training data, model weights, and documentation. The survey highlights the need for more diverse dialectal datasets and attributes the importance of openness for research reproducibility and transparency. It concludes by identifying key challenges and opportunities for future research and stressing the need for more inclusive and representative models.
Submitted to ACM Transactions on Asian and Low-Resource Language Information Processing
References in corpus (13)
- LLaMA: Open and Efficient Foundation Language Models
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- GPT-4o System Card
- Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models
- AraPoemBERT: A Pretrained Language Model for Arabic Poetry Analysis
- Fanar: An Arabic-Centric Multimodal Generative AI Platform
- EgyBERT: A Large Language Model Pretrained on Egyptian Dialect Corpora
- ArabianGPT: Native Arabic GPT-based Large Language Model
- CamelEval: Advancing Culturally Aligned Arabic Language Models and Benchmarks
- TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varieties
- Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion
- Kuwain 1.5B: An Arabic SLM via Language Injection
- Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic