Entropic Inference: some pitfalls and paradoxes we can avoid

Abstract

The method of maximum entropy has been very successful but there are cases where it has either failed or led to paradoxes that have cast doubt on its general legitimacy. My more optimistic assessment is that such failures and paradoxes provide us with valuable learning opportunities to sharpen our skills in the proper way to deploy entropic methods. The central theme of this paper revolves around the different ways in which constraints are used to capture the information that is relevant to a problem. This leads us to focus on four epistemically different types of constraints. I propose that the failure to recognize the distinctions between them is a prime source of errors. I explicitly discuss two examples. One concerns the dangers involved in replacing expected values with sample averages. The other revolves around misunderstanding ignorance. I discuss the Friedman-Shimony paradox as it is manifested in the three-sided die problem and also in its original thermodynamic formulation.

14 pages, 1 figure. Invited paper presented at MaxEnt 2012, The 32nd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, (July 15--20, 2012, Garching, Germany)

References in corpus (1)

• Updating Probabilities

Cited by in corpus (6)

• Cell responses only partially shape cell-to-cell variations in protein abundances in Escherichia coli chemotaxis
• On Entropy Regularized Path Integral Control for Trajectory Optimization
• Data-driven quantification of robustness and sensitivity of cell signaling networks
• A new signature of quantum phase transitions from the numerical range
• Inferring transition rates on networks with incomplete knowledge
• The Census and the Second Law: An Entropic Approach to Optimal Apportionment for the U.S. House of Representatives