Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction

Abstract

Discovering relevant, but possibly hidden, variables is a key step in constructing useful and predictive theories about the natural world. This brief note explains the connections between three approaches to this problem: the recently introduced information-bottleneck method, the computational mechanics approach to inferring optimal models, and Salmon's statistical relevance basis.

3 pages, no figures, submitted to PRE as a "brief report". Revision: added an acknowledgements section originally omitted by a LaTeX bug

References in corpus (2)

• The information bottleneck method
• Computational Mechanics: Pattern and Prediction, Structure and Simplicity

Cited by in corpus (9)

• An Algorithm for Pattern Discovery in Time Series
• Computational Mechanics of Input-Output Processes: Structured transformations and the $\epsilon$-transducer
• Informational and Causal Architecture of Discrete-Time Renewal Processes
• Optimal high-level descriptions of dynamical systems
• Information theory and learning: a physical approach
• An Information-Theoretic Formalism for Multiscale Structure in Complex Systems
• Circumventing the Curse of Dimensionality in Prediction: Causal Rate-Distortion for Infinite-Order Markov Processes
• Finite-State Extreme Effect Variable
• Understanding and Designing Complex Systems: Response to "A framework for optimal high-level descriptions in science and engineering---preliminary report"