Showing cs.LGShow all
3 papers · 1 filter
cs.LG2023
Empirical Loss Landscape Analysis of Neural Network Activation Functions
Anna Sergeevna Bosman, Andries Engelbrecht, Marde Helbig
Activation functions play a significant role in neural network design by enabling non-linearity. The choice of activation function was previously shown to influence the properties…
cs.LG2019
Loss Surface Modality of Feed-Forward Neural Network Architectures
Anna Sergeevna Bosman, Andries Engelbrecht, Mardé Helbig
It has been argued in the past that high-dimensional neural networks do not exhibit local minima capable of trapping an optimisation algorithm. However, the relationship between lo…
cs.LG2019
Visualising Basins of Attraction for the Cross-Entropy and the Squared Error Neural Network Loss Functions
Anna Sergeevna Bosman, Andries Engelbrecht, Mardé Helbig
Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural networ…