DSG-KD: Knowledge Distillation from Domain-Specific to General Language Models

Sangyeon Cho, Jangyeong Jeon, Dongjoon Lee +2

The use of pre-trained language models fine-tuned to address specific downstream tasks is a common approach in natural language processing (NLP). However, acquiring domain-specific…