Information Loss in Static Nonlinearities

Abstract

In this work, conditional entropy is used to quantify the information loss induced by passing a continuous random variable through a memoryless nonlinear input-output system. We derive an expression for the information loss depending on the input density and the nonlinearity and show that the result is strongly related to the non-injectivity of the considered system. Tight upper bounds are presented, which can be evaluated with less difficulty than a direct evaluation of the information loss, which involves the logarithm of a sum. Application of our results is illustrated on a set of examples.

9 pages, 6 figures; A short version of this paper is submitted to an IEEE conference

References in corpus (1)

• Information Loss in Static Nonlinearities

Cited by in corpus (6)

• Relative Information Loss in the PCA
• Signal Enhancement as Minimization of Relevant Information Loss
• Information Loss in Static Nonlinearities
• Some Results on the Information Loss in Dynamical Systems
• Renormalized Mutual Information for Artificial Scientific Discovery
• Information Measures for Deterministic Input-Output Systems