Circuit design in biology and machine learning. I. Random networks and dimensional reduction

Abstract

A biological circuit is a neural or biochemical cascade, taking inputs and producing outputs. How have biological circuits learned to solve environmental challenges over the history of life? The answer certainly follows Dobzhansky's famous quote that ``nothing in biology makes sense except in the light of evolution.'' But that quote leaves out the mechanistic basis by which natural selection's trial-and-error learning happens, which is exactly what we have to understand. How does the learning process that designs biological circuits actually work? How much insight can we gain about the form and function of biological circuits by studying the processes that have made those circuits? Because life's circuits must often solve the same problems as those faced by machine learning, such as environmental tracking, homeostatic control, dimensional reduction, or classification, we can begin by considering how machine learning designs computational circuits to solve problems. We can then ask: How much insight do those computational circuits provide about the design of biological circuits? How much does biology differ from computers in the particular circuit designs that it uses to solve problems? This article steps through two classic machine learning models to set the foundation for analyzing broad questions about the design of biological circuits. One insight is the surprising power of randomly connected networks. Another is the central role of internal models of the environment embedded within biological circuits, illustrated by a model of dimensional reduction and trend prediction. Overall, many challenges in biology have machine learning analogs, suggesting hypotheses about how biology's circuits are designed.

Added background info in two text boxes and new figure, edited throughout

References in corpus (14)

• Next Generation Reservoir Computing
• A survey of dimensionality reduction techniques
• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• Evolution of bow-tie architectures in biology
• Maladaptation and the paradox of robustness in evolution
• Dimensional Reduction in Complex Living Systems: Where, Why, and How
• Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
• Reservoir Computing for Macroeconomic Forecasting with Mixed Frequency Data
• Robustness and complexity
• Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems
• ReservoirComputing.jl: An Efficient and Modular Library for Reservoir Computing Models
• The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold
• Canalisation and plasticity on the developmental manifold of Caenorhabditis elegans
• A biological circuit to anticipate trend