200 citations · 675 across the 45 of their papers we have counts for
26 papers · 1 filter
Information Plane Analysis for Dropout Neural Networks
Linara Adilova, Bernhard C. Geiger, Asja Fischer
The information-theoretic framework promises to explain the predictive power of neural networks. In particular, the information plane analysis, which measures mutual information (M…
Information-Theoretic Reduction of Markov Chains
Bernhard C. Geiger
We survey information-theoretic approaches to the reduction of Markov chains. Our survey is structured in two parts: The first part considers Markov chain coarse graining, which fo…
On Functions of Markov Random Fields
Bernhard C. Geiger, Ali Al-Bashabsheh
We derive two sufficient conditions for a function of a Markov random field (MRF) on a given graph to be a MRF on the same graph. The first condition is information-theoretic and p…
A Short Note on the Jensen-Shannon Divergence between Simple Mixture Distributions
Bernhard C. Geiger
This short note presents results about the symmetric Jensen-Shannon divergence between two discrete mixture distributions and . Specifically, for , is the m…
On the Information Dimension of Multivariate Gaussian Processes
Bernhard C. Geiger, Tobias Koch
The authors have recently defined the Rényi information dimension rate of a stationary stochastic process as the entropy rate of the uniform…
On the Information Dimension of Stochastic Processes
Bernhard C. Geiger, Tobias Koch
In 1959, Rényi proposed the information dimension and the -dimensional entropy to measure the information content of general random variables. This paper proposes a generalizati…
Divergence Scaling of Fixed-Length, Binary-Output, One-to-One Distribution Matching
Patrick Schulte, Bernhard C. Geiger
Distribution matching is the process of invertibly mapping a uniformly distributed input sequence onto sequences that approximate the output of a desired discrete memoryless source…
A Rate-Distortion Approach to Caching
Roy Timo, Shirin Saeedi Bidokhti, Michèle Wigger +1
This paper takes a rate-distortion approach to understanding the information-theoretic laws governing cache-aided communications systems. Specifically, we characterise the optimal…
Hard Clusters Maximize Mutual Information
Bernhard C. Geiger, Rana Ali Amjad
In this paper, we investigate mutual information as a cost function for clustering, and show in which cases hard, i.e., deterministic, clusters are optimal. Using convexity propert…
Higher-Order Kullback-Leibler Aggregation of Markov Chains
Bernhard C. Geiger, Yuchen Wu
We consider the problem of reducing a first-order Markov chain on a large alphabet to a higher-order Markov chain on a small alphabet. We present information-theoretic cost functio…
Greedy Algorithms for Optimal Distribution Approximation
Bernhard C. Geiger, Georg Böcherer
The approximation of a discrete probability distribution by an -type distribution is considered. The approximation error is measured by the information…
Graph-Based Lossless Markov Lumpings
Bernhard C. Geiger, Christoph Hofer-Temmel
We use results from zero-error information theory to determine the set of non-injective functions through which a Markov chain can be projected without losing information. These lu…
The Fractality of Polar and Reed-Muller Codes
Bernhard C. Geiger
The generator matrices of polar codes and Reed-Muller codes are obtained by selecting rows from the Kronecker product of a lower-triangular binary square matrix. For polar codes, t…
Information Loss and Anti-Aliasing Filters in Multirate Systems
Bernhard C. Geiger, Gernot Kubin
This work investigates the information loss in a decimation system, i.e., in a downsampler preceded by an anti-aliasing filter. It is shown that, without a specific signal model in…
Optimal Quantization for Distribution Synthesis
Georg Böcherer, Bernhard C. Geiger
Finite precision approximations of discrete probability distributions are considered, applicable for distribution synthesis, e.g., probabilistic shaping. Two algorithms are present…
On the Rate of Information Loss in Memoryless Systems
Bernhard C. Geiger, Gernot Kubin
In this work we present results about the rate of (relative) information loss induced by passing a real-valued, stationary stochastic process through a memoryless system. We show t…
Information Measures for Deterministic Input-Output Systems
Bernhard C. Geiger, Gernot Kubin
In this work the information loss in deterministic, memoryless systems is investigated by evaluating the conditional entropy of the input random variable given the output random va…
Information-Preserving Markov Aggregation
Bernhard C. Geiger, Christoph Temmel
We present a sufficient condition for a non-injective function of a Markov chain to be a second-order Markov chain with the same entropy rate as the original chain. This permits an…
Lumpings of Markov chains, entropy rate preservation, and higher-order lumpability
Bernhard C. Geiger, Christoph Temmel
A lumping of a Markov chain is a coordinate-wise projection of the chain. We characterise the entropy rate preservation of a lumping of an aperiodic and irreducible Markov chain on…
Signal Enhancement as Minimization of Relevant Information Loss
Bernhard C. Geiger, Gernot Kubin
We introduce the notion of relevant information loss for the purpose of casting the signal enhancement problem in information-theoretic terms. We show that many algorithms from mac…
Relative Information Loss in the PCA
Bernhard C. Geiger, Gernot Kubin
In this work we analyze principle component analysis (PCA) as a deterministic input-output system. We show that the relative information loss induced by reducing the dimensionality…
Relative Information Loss - An Introduction
Bernhard C. Geiger, Gernot Kubin
We introduce a relative variant of information loss to characterize the behavior of deterministic input-output systems. We show that the relative loss is closely related to Renyi's…
On the Information Loss in Memoryless Systems: The Multivariate Case
Bernhard C. Geiger, Gernot Kubin
In this work we give a concise definition of information loss from a system-theoretic point of view. Based on this definition, we analyze the information loss in static input-outpu…
Influence of Doppler Bin Width on GPS Acquisition Probabilities
Bernhard C. Geiger, Christian Vogel
Acquisition is a search in two continuous dimensions, where the digital algorithms require a partitioning of the search space into cells. Depending on the partitioning of the Doppl…