Noise mitigation strategies in physical feedforward neural networks

Abstract

Physical neural networks are promising candidates for next generation artificial intelligence hardware. In such architectures, neurons and connections are physically realized and do not leverage digital concepts with their practically infinite signal-to-noise ratio to encode, transduce and transform information. They therefore are prone to noise with a variety of statistical and architectural properties, and effective strategies leveraging network-inherent assets to mitigate noise in an hardware-efficient manner are important in the pursuit of next generation neural network hardware. Based on analytical derivations, we here introduce and analyse a variety of different noise-mitigation approaches. We analytically show that intra-layer connections in which the connection matrix's squared mean exceeds the mean of its square fully suppresses uncorrelated noise. We go beyond and develop two synergistic strategies for noise that is uncorrelated and correlated across populations of neurons. First, we introduce the concept of ghost neurons, where each group of neurons perturbed by correlated noise has a negative connection to a single neuron, yet without receiving any input information. Secondly, we show that pooling of neuron populations is an efficient approach to suppress uncorrelated noise. As such, we developed a general noise mitigation strategy leveraging the statistical properties of the different noise terms most relevant in analogue hardware. Finally, we demonstrate the effectiveness of this combined approach for trained neural network classifying the MNIST handwritten digits, for which we achieve a 4-fold improvement of the output signal-to-noise ratio and increase the classification accuracy almost to the level of the noise-free network.

References in corpus (10)

• Deep Learning with Coherent Nanophotonic Circuits
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Parallel convolution processing using an integrated photonic tensor core
• Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems
• Reinforcement Learning in a large scale photonic Recurrent Neural Network
• An optical neural network using less than 1 photon per multiplication
• Three dimensional waveguide-interconnects for scalable integration of photonic neural networks
• Understanding and mitigating noise in trained deep neural networks
• Fundamental aspects of noise in analog-hardware neural networks
• Optical Neural Networks: The 3D connection

Cited by in corpus (1)

• Multiplexing-based control of stochastic resonance