Bits from Biology for Computational Intelligence

Abstract

Computational intelligence is broadly defined as biologically-inspired computing. Usually, inspiration is drawn from neural systems. This article shows how to analyze neural systems using information theory to obtain constraints that help identify the algorithms run by such systems and the information they represent. Algorithms and representations identified information-theoretically may then guide the design of biologically inspired computing systems (BICS). The material covered includes the necessary introduction to information theory and the estimation of information theoretic quantities from neural data. We then show how to analyze the information encoded in a system about its environment, and also discuss recent methodological developments on the question of how much information each agent carries about the environment either uniquely, or redundantly or synergistically together with others. Last, we introduce the framework of local information dynamics, where information processing is decomposed into component processes of information storage, transfer, and modification -- locally in space and time. We close by discussing example applications of these measures to neural data and other complex systems.

References in corpus (6)

• JIDT: An information-theoretic toolkit for studying the dynamics of complex systems
• Local information transfer as a spatiotemporal filter for complex systems
• Nonnegative Decomposition of Multivariate Information
• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• Synergy and redundancy in the Granger causal analysis of dynamical networks
• Identification of functional information subgraphs in complex networks

Cited by in corpus (22)

• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions
• Control of criticality and computation in spiking neuromorphic networks with plasticity
• Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes
• Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic
• Breakdown of local information processing may underlie isoflurane anesthesia effects
• Introducing a differentiable measure of pointwise shared information
• Quantifying synergistic information using intermediate stochastic variables
• A comparison of partial information decompositions using data from real and simulated layer 5b pyramidal cells
• Secret Sharing and Shared Information
• Tackling the subsampling problem to infer collective properties from limited data
• MR. Estimator, a toolbox to determine intrinsic timescales from subsampled spiking activity
• A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions
• Optimal interplay between synaptic strengths and network structure enhances activity fluctuations and information propagation in hierarchical modular networks
• Can Transfer Entropy Infer Information Flow in Neuronal Circuits for Cognitive Processing?
• The identity of information: how deterministic dependencies constrain information synergy and redundancy
• Decomposing information into copying versus transformation
• MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition
• Contrasting information theoretic decompositions of modulatory and arithmetic interactions in neural information processing systems
• Quantifying multivariate redundancy with maximum entropy decompositions of mutual information
• Information theoretic analysis of computational models as a tool to understand the neural basis of behaviors
• Towards an information theoretical description of communication in brain networks