4 papers
Isotropic Activation Functions Enable Deindividuated Neurons and Adaptive Topologies
George Bird
Introduced is a methodology for adapting the topology of dense neural networks, enabled by isotropic activation functions. Achieved through prescribed reparameterisation symmetries…
The Affine Divergence: Aligning Activation Updates Beyond Normalisation
George Bird
A systematic mismatch exists between mathematically ideal and effective activation updates during gradient descent. As intended, parameters update in their direction of steepest de…
Emergence of Quantised Representations Isolated to Anisotropic Functions
George Bird
Presented is a novel methodology for determining representational structure, which builds upon the existing Spotlight Resonance method. This new tool is used to gain insight into h…
The Spotlight Resonance Method: Resolving the Alignment of Embedded Activations
George Bird
Understanding how deep learning models represent data is currently difficult due to the limited number of methodologies available. This paper demonstrates a versatile and novel vis…