Signal Enhancement as Minimization of Relevant Information Loss

Abstract

We introduce the notion of relevant information loss for the purpose of casting the signal enhancement problem in information-theoretic terms. We show that many algorithms from machine learning can be reformulated using relevant information loss, which allows their application to the aforementioned problem. As a particular example we analyze principle component analysis for dimensionality reduction, discuss its optimality, and show that the relevant information loss can indeed vanish if the relevant information is concentrated on a lower-dimensional subspace of the input space.

9 pages; 4 figures; accepted for presentation at a conference

References in corpus (3)

• Relative Information Loss in the PCA
• Information Loss in Static Nonlinearities
• Relative Information Loss - An Introduction

Cited by in corpus (3)

• Relative Information Loss in the PCA
• Quantitative analysis of spectroscopic Low Energy Electron Microscopy data: High-dynamic range imaging, drift correction and cluster analysis
• Information Measures for Deterministic Input-Output Systems