Desynchronous Learning in a Physics-Driven Learning Network

Abstract

In a neuron network, synapses update individually using local information, allowing for entirely decentralized learning. In contrast, elements in an artificial neural network (ANN) are typically updated simultaneously using a central processor. Here we investigate the feasibility and effect of desynchronous learning in a recently introduced decentralized, physics-driven learning network. We show that desynchronizing the learning process does not degrade performance for a variety of tasks in an idealized simulation. In experiment, desynchronization actually improves performance by allowing the system to better explore the discretized state space of solutions. We draw an analogy between desynchronization and mini-batching in stochastic gradient descent, and show that they have similar effects on the learning process. Desynchronizing the learning process establishes physics-driven learning networks as truly fully distributed learning machines, promoting better performance and scalability in deployment.

arXiv comment6 pages 4 figures

References in corpus (14)

• A high-bias, low-variance introduction to Machine Learning for physicists
• Memory formation in matter
• Designing allostery-inspired response in mechanical networks
• Network Plasticity as Bayesian Inference
• Multiple transient memories in experiments on sheared non-Brownian suspensions
• Generic transient memory formation in disordered systems with noise
• Directed aging, memory and Nature's greed
• Supervised learning in a mechanical system
• Supervised learning in physical networks: From machine learning to learning machines
• Periodic training of creeping solids
• Shaping dynamical folding and misfolding pathways in mechanical metamaterials
• Effect of aging on the non-linear elasticity and memory formation in materials
• Learned multi-stability in mechanical networks
• Local rules for fabricating allosteric networks

Cited by in corpus (13)

• Learning without neurons in physical systems
• Demonstration of Decentralized, Physics-Driven Learning
• Soft Metamaterials: Adaptation and Intelligence
• Machine Learning Without a Processor: Emergent Learning in a Nonlinear Electronic Metamaterial
• The Physical Effects of Learning
• Learning to self-fold at a bifurcation
• Training Coupled Phase Oscillators as a Neuromorphic Platform using Equilibrium Propagation
• Rigidity of Epithelial Tissues as a Double Optimization Problem
• Physical Neural Networks with Self-Learning Capabilities
• Self-Contrastive Forward-Forward Algorithm
• Physical networks become what they learn
• A universal approximation theorem for nonlinear resistive networks
• Physical learning beyond the quasistatic limit