8.1k citations · 11.3k across the 10 of their papers we have counts for
Showing cs.ITShow all
2 papers · 1 filter
cs.IT2014★ 72 cited
Notes on Kullback-Leibler Divergence and Likelihood
Jonathon Shlens
The Kullback-Leibler (KL) divergence is a fundamental equation of information theory that quantifies the proximity of two probability distributions. Although difficult to understan…
cs.IT2014★ 2 cited
A Light Discussion and Derivation of Entropy
Jonathon Shlens
The expression for entropy sometimes appears mysterious - as it often is asserted without justification. This short manuscript contains a discussion of the underlying assumptions b…