A Survey of Classical And Quantum Sequence Models

Abstract

Our primary objective is to conduct a brief survey of various classical and quantum neural net sequence models, which includes self-attention and recurrent neural networks, with a focus on recent quantum approaches proposed to work with near-term quantum devices, while exploring some basic enhancements for these quantum models. We re-implement a key representative set of these existing methods, adapting an image classification approach using quantum self-attention to create a quantum hybrid transformer that works for text and image classification, and applying quantum self-attention and quantum recurrent neural networks to natural language processing tasks. We also explore different encoding techniques and introduce positional encoding into quantum self-attention neural networks leading to improved accuracy and faster convergence in text and image classification experiments. This paper also performs a comparative analysis of classical self-attention models and their quantum counterparts, helping shed light on the differences in these models and their performance.

arXiv comment6 pages, 10 figures, accepted as a COMSNETS paper

References in corpus (14)

• Sequence to Sequence Learning with Neural Networks
• Supervised learning with quantum enhanced feature spaces
• Parameterized quantum circuits as machine learning models
• Circuit-centric quantum classifiers
• The effect of data encoding on the expressive power of variational quantum machine learning models
• Quantum advantage in learning from experiments
• Quantum autoencoders for efficient compression of quantum data
• Quantum Generative Adversarial Networks for Learning and Loading Random Distributions
• Recurrent Neural Networks (RNNs): A gentle Introduction and Overview
• QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer
• Learning temporal data with variational quantum recurrent neural network
• Rapid training of quantum recurrent neural networks
• Quantum Self-Attention Neural Networks for Text Classification
• QSAN: A Near-term Achievable Quantum Self-Attention Network