Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
MeRoTune: RoPE-Safe Merging with a Tunable Dial
Salman Faroz
When you merge two fine-tuned models from the same base checkpoint by simply averaging their weights, you implicitly assume their attention subspaces are still aligned. Recent work…
cs.CL2025
Domain-Adaptive Continued Pre-Training of Small Language Models
Salman Faroz
Continued pre-training of small language models offers a promising path for domain adaptation with limited computational resources. I've investigated this approach within education…
cs.CL2024
DisEmbed: Transforming Disease Understanding through Embeddings
Salman Faroz
The medical domain is vast and diverse, with many existing embedding models focused on general healthcare applications. However, these models often struggle to capture a deep under…