4 papers
Isotropic Activation Functions Enable Deindividuated Neurons and Adaptive Topologies
George Bird
Varying the number of neurons while preserving function is difficult because neural networks' dense connectivity creates strong computational interdependence between units. Element…
The Affine Divergence: Aligning Activation Updates Beyond Normalisation
George Bird
A systematic mismatch exists between mathematically ideal and effective activation updates during gradient descent. As intended, parameters update in their direction of steepest de…
Emergence of Quantised Representations Isolated to Anisotropic Functions
George Bird
Presented is a novel methodology for determining representational structure, which builds upon the existing Spotlight Resonance method. This new tool is used to gain insight into h…
The Spotlight Resonance Method: Resolving the Alignment of Embedded Activations
George Bird
Understanding how deep learning models represent data is currently difficult due to the limited number of methodologies available. This paper demonstrates a versatile and novel vis…