Knowledge-Instruct: Effective Continual Pre-training from Limited Data using Instructions

Oded Ovadia, Meni Brief, Rachel Lemberg +1

While Large Language Models (LLMs) acquire vast knowledge during pre-training, they often lack domain-specific, new, or niche information. Continual pre-training (CPT) attempts to…