9.7k citations · 26.5k across the 43 of their papers we have counts for
Showing 2014 · cs.ITShow all
2 papers · 2 filters
cs.IT2014★ 72 cited
Notes on Kullback-Leibler Divergence and Likelihood
Jonathon Shlens
The Kullback-Leibler (KL) divergence is a fundamental equation of information theory that quantifies the proximity of two probability distributions. Although difficult to understan…
cs.IT2014★ 2 cited
A Light Discussion and Derivation of Entropy
Jonathon Shlens
The expression for entropy sometimes appears mysterious - as it often is asserted without justification. This short manuscript contains a discussion of the underlying assumptions b…