Domain-Adaptive Continued Pre-Training of Small Language Models

Salman Faroz

Continued pre-training of small language models offers a promising path for domain adaptation with limited computational resources. I've investigated this approach within education…