The Condition Number as a Scale-Invariant Proxy for Information Encoding in Neural Units

Oswaldo Ludwig

This paper explores the relationship between the condition number of a neural network's weight tensor and the extent of information encoded by the associated processing unit, viewe…