Finding Skill Neurons in Pre-trained Transformer-based Language Models

Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang +3

Transformer-based pre-trained language models have demonstrated superior performance on various natural language processing tasks. However, it remains unclear how the skills requir…