On Relation-Specific Neurons in Large Language Models

Yihong Liu, Runsheng Chen, Lea Hirlimann +6

In large language models (LLMs), certain \emph{neurons} can store distinct pieces of knowledge learned during pretraining. While factual knowledge typically appears as a combinatio…