9 papers
YaPO: Learnable Sparse Activation Steering Vectors for Domain Adaptation
Abdelaziz Bounhar, Rania Hossam Elmohamady Elbadry, Hadi Abdine +3
Steering Large Language Models (LLMs) through activation interventions has emerged as a lightweight alternative to fine-tuning for alignment and personalization. Recent work on Bi-…
MixtureKit: A General Framework for Composing, Training, and Visualizing Mixture-of-Experts Models
Ahmad Chamma, Omar El Herraoui, Guokan Shang
We introduce MixtureKit, a modular open-source framework for constructing, training, and analyzing Mixture-of-Experts (MoE) models from arbitrary pre-trained or fine-tuned models.…
Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules
Amr Mohamed, Yang Zhang, Michalis Vazirgiannis +1
Diffusion large language models (dLLMs) offer a promising alternative to autoregressive models, but their practical utility is severely hampered by slow, iterative sampling. We pre…
Shorter but not Worse: Frugal Reasoning via Easy Samples as Length Regularizers in Math RLVR
Abdelaziz Bounhar, Hadi Abdine, Evan Dufraisse +5
Large language models (LLMs) trained for step-by-step reasoning often become excessively verbose, raising inference cost. Standard Reinforcement Learning with Verifiable Rewards (R…
Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts
Guokan Shang, Hadi Abdine, Ahmad Chamma +7
We introduce Nile-Chat-4B, 3x4B-A6B, and 12B, a collection of LLMs for Egyptian dialect, uniquely designed to understand and generate texts written in both Arabic and Latin scripts…
Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text
Amr Mohamed, Yang Zhang, Michalis Vazirgiannis +1
Code-switching (CSW) is the act of alternating between two or more languages within a single discourse. This phenomenon is widespread in multilingual communities, and increasingly…