Polysemanticity and Capacity in Neural Networks

Adam Scherlis, Kshitij Sachan, Adam S. Jermyn +2

Individual neurons in neural networks often represent a mixture of unrelated features. This phenomenon, called polysemanticity, can make interpreting neural networks more difficult…