Recurrent Quantum Neural Networks

Abstract

Recurrent neural networks are the foundation of many sequence-to-sequence models in machine learning, such as machine translation and speech synthesis. In contrast, applied quantum computing is in its infancy. Nevertheless there already exist quantum machine learning models such as variational quantum eigensolvers which have been used successfully e.g. in the context of energy minimization tasks. In this work we construct a quantum recurrent neural network (QRNN) with demonstrable performance on non-trivial tasks such as sequence learning and integer digit classification. The QRNN cell is built from parametrized quantum neurons, which, in conjunction with amplitude amplification, create a nonlinear activation of polynomials of its inputs and cell state, and allow the extraction of a probability distribution over predicted classes at each step. To study the model's performance, we provide an implementation in pytorch, which allows the relatively efficient optimization of parametrized quantum circuits with thousands of parameters. We establish a QRNN training setup by benchmarking optimization hyperparameters, and analyse suitable network topologies for simple memorisation and sequence prediction tasks from Elman's seminal paper (1990) on temporal structure learning. We then proceed to evaluate the QRNN on MNIST classification, both by feeding the QRNN each image pixel-by-pixel; and by utilising modern data augmentation as preprocessing step. Finally, we analyse to what extent the unitary nature of the network counteracts the vanishing gradient problem that plagues many existing quantum classifiers and classical RNNs.

22 pages

References in corpus (17)

• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• A variational eigenvalue solver on a quantum processor
• Quantum algorithm for solving linear systems of equations
• The theory of variational hybrid quantum-classical algorithms
• Reformer: The Efficient Transformer
• Accelerated Variational Quantum Eigensolver
• Strategies for solving the Fermi-Hubbard model on near-term quantum computers
• Quantum Hopfield neural network
• On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
• Quantum Neuron: an elementary building block for machine learning on quantum computers
• Full-Capacity Unitary Recurrent Neural Networks
• Using a Recurrent Neural Network to Reconstruct Quantum Dynamics of a Superconducting Qubit from Physical Observations
• Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context
• Majorana loop stabilizer codes for error correction of fermionic quantum simulations
• Quantum classification of the MNIST dataset with Slow Feature Analysis
• Training Optimization for Gate-Model Quantum Neural Networks
• Adaptively Truncating Backpropagation Through Time to Control Gradient Bias

Cited by in corpus (7)

• Variational Quantum Reinforcement Learning via Evolutionary Optimization
• Quantum Architecture Search via Deep Reinforcement Learning
• Scrambling Ability of Quantum Neural Networks Architectures
• A review of Quantum Neural Networks: Methods, Models, Dilemma
• Hybrid Quantum-Classical Graph Convolutional Network
• Towards Efficient Ansatz Architecture for Variational Quantum Algorithms
• On quantum neural networks