36 citations · 85 across the 18 of their papers we have counts for
Showing 2019Show all
2 papers · 1 filter
cs.LG2019★ 36 cited
Rethinking Softmax with Cross-Entropy: Neural Network Classifier as Mutual Information Estimator
Zhenyue Qin, Dongwoo Kim, Tom Gedeon
Mutual information is widely applied to learn latent representations of observations, whilst its implication in classification neural networks remain to be better explained. We sho…
cs.LG2019★ 2 cited
Softmax Is Not an Artificial Trick: An Information-Theoretic View of Softmax in Neural Networks
Zhenyue Qin, Dongwoo Kim
Despite great popularity of applying softmax to map the non-normalised outputs of a neural network to a probability distribution over predicting classes, this normalised exponentia…