Mix-CPT: A Domain Adaptation Framework via Decoupling Knowledge Learning and Format Alignment

Jinhao Jiang, Junyi Li, Wayne Xin Zhao +3

Adapting general large language models (LLMs) to specialized domains presents great challenges due to varied data distributions. This adaptation typically requires continual pre-tr…