235 citations · 235 across the 4 of their papers we have counts for
4 papers
MateICL: Mitigating Attention Dispersion in Large-Scale In-Context Learning
Murtadha Ahmed, Wenbo, Liu yunfeng
Large Language Models (LLMs) have demonstrated remarkable capabilities in In-Context Learning (ICL). However, the fixed position length constraints in pre-trained models limit the…
AlcLaM: Arabic Dialectal Language Model
Murtadha Ahmed, Saghir Alfasly, Bo Wen +3
Pre-trained Language Models (PLMs) are integral to many modern natural language processing (NLP) systems. Although multilingual models cover a wide range of languages, they often g…
Naive Bayes-based Context Extension for Large Language Models
Jianlin Su, Murtadha Ahmed, Wenbo +3
Large Language Models (LLMs) have shown promising in-context learning abilities. However, conventional In-Context Learning (ICL) approaches are often impeded by length limitations…
RoFormer: Enhanced Transformer with Rotary Position Embedding
Jianlin Su, Yu Lu, Shengfeng Pan +3
Position encoding recently has shown effective in the transformer architecture. It enables valuable supervision for dependency modeling between elements at different positions of t…