Rate of mutual information between coarse-grained non-Markovian variables

Abstract

The problem of calculating the rate of mutual information between two coarse-grained variables that together specify a continuous time Markov process is addressed. As a main obstacle, the coarse-grained variables are in general non-Markovian, therefore, an expression for their Shannon entropy rates in terms of the stationary probability distribution is not known. A numerical method to estimate the Shannon entropy rate of continuous time hidden-Markov processes from a single time series is developed. With this method the rate of mutual information can be determined numerically. Moreover, an analytical upper bound on the rate of mutual information is calculated for a class of Markov processes for which the transition rates have a bipartite character. Our general results are illustrated with explicit calculations for four-state networks.

17 pages, 7 figures

References in corpus (10)

• Dissipation: The phase-space perspective
• Work and information processing in a solvable model of Maxwell's demon
• Thermodynamics of a physical model implementing a Maxwell demon
• Nonequilibrium Detailed Fluctuation Theorem for Repeated Discrete Feedback
• Extracting work from a single heat bath through feedback
• Thermodynamics of feedback controlled systems
• Stochastic thermodynamics for "Maxwell demon" feedbacks
• Fluctuation relations and coarse-graining
• Information-theoretic vs. thermodynamic entropy production in autonomous sensory networks
• Efficiency of a Brownian information machine

Cited by in corpus (23)

• Thermodynamics with continuous information flow
• Stochastic thermodynamics of bipartite systems: transfer entropy inequalities and a Maxwell's demon interpretation
• Efficiency of cellular information processing
• Information Thermodynamics on Causal Networks
• Cost and Precision of Brownian Clocks
• Second-law-like inequalities with information and their interpretations
• Theory of nonequilibrium free energy transduction by molecular machines
• Sensory capacity: an information theoretical measure of the performance of a sensor
• Thermodynamic limits to information harvesting by sensory systems
• Thermodynamic cost of external control
• Energy and information flows in autonomous systems
• Information thermodynamics for interacting stochastic systems without bipartite structure
• Thermodynamic aspects of information transfer in complex dynamical systems
• Total cost of operating an information engine
• Role of sufficient statistics in stochastic thermodynamics and its implication to sensory adaptation
• Stochastic thermodynamics based on incomplete information: Generalized Jarzynski equality with measurement errors with or without feedback
• Dispersion for Two Classes of Random Variables: General Theory and Application to Inference of an External Ligand Concentration by a Cell
• Information Landscape and Flux, Mutual Information Rate Decomposition and Entropy Production
• Information Thermodynamics of the Transition-Path Ensemble
• Coupled-Double-Quantum-Dot Environmental Information Engines: A Numerical Analysis
• Application of Mutual Information Methods in Time-Distance Helioseismology
• Feedback-charging a metallic island
• Mutual Information in Coupled Double Quantum Dots: A Simple Analytic Model for Potential Artificial Consciousness