Nonadditive Entropies Yield Probability Distributions with Biases not Warranted by the Data

Abstract

Different quantities that go by the name of entropy are used in variational principles to infer probability distributions from limited data. Shore and Johnson showed that maximizing the Boltzmann- Gibbs form of the entropy ensures that probability distributions inferred satisfy the multiplication rule of probability for independent events in the absence of data coupling such events. Other types of entropies that violate the Shore and Johnson axioms, including nonadditive entropies such as the Tsallis entropy, violate this basic consistency requirement. Here we use the axiomatic framework of Shore and Johnson to show how such nonadditive entropy functions generate biases in probability distributions that are not warranted by the underlying data.

References in corpus (1)

• On the robustness of q-expectation values and Renyi entropy

Cited by in corpus (31)

• Maximum Entropy Principle in statistical inference: case for non-Shannonian entropies
• Entropy, Information, and the Updating of Probabilities
• An overview of generalized entropic forms
• When Shannon and Khinchin meet Shore and Johnson: equivalence of information theory and statistical inference axiomatics
• Thermodynamics is more powerful than the role to it reserved by Boltzmann-Gibbs statistical mechanics
• Random versus maximum entropy models of neural population activity
• Time evolution of Renyi entropy under the Lindblad equation
• Conceptual Inadequacy of the Shore and Johnson Axioms for Wide Classes of Complex Systems
• Rényi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data
• Thermodynamics from first principles: correlations and nonextensivity
• Stochastic thermodynamics and fluctuation theorems for non-linear systems
• A Collision Operator for Describing Dissipation in Noncanonical Phase Space
• Comment on "Black Hole Entropy: A Closer Look"
• An information and field theoretic approach to the grand canonical ensemble
• A Bayesian interpretation of abrupt phase transitions
• Dimensional measures of generalized entropy
• A superstatistical measure of distance from canonical equilibrium
• Entropic Dynamics of Networks
• Reply to "Comment on `Rényi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data' "
• Comment on "Rényi entropy yields artifficial biases not in the data and incorrect updating due to the infinite-size data"
• Entropic analysis of the localization-delocalization transition in a one-dimensional correlated lattice
• Exponential Family Models from Bayes' Theorem under Expectation Constraints
• Strong correlations between the exponent $\alpha$ and the particle number for a Renyi-monoatomic gas in Gibbs' statistical mechanics
• Nonparametric Maximum Entropy Probability Density Estimation
• Simplified calculations of plasma oscillations with non-extensive statistics
• Kappa distributions in the framework of superstatistics
• Symmetry and the salience of textures
• The internal energy expression of a long-range interaction complex system and its statistical physical properties
• The expression of ensemble average internal energy in long-range interaction complex system and its statistical physical properties
• An approach to the quasi-equilibrium state of a self-gravitating system
• Adaptive Decision Making via Entropy Minimization