BCQQ: Batch-Constraint Quantum Q-Learning with Cyclic Data Re-uploading

Abstract

Deep reinforcement learning (DRL) often requires a large number of data and environment interactions, making the training process time-consuming. This challenge is further exacerbated in the case of batch RL, where the agent is trained solely on a pre-collected dataset without environment interactions. Recent advancements in quantum computing suggest that quantum models might require less data for training compared to classical methods. In this paper, we investigate this potential advantage by proposing a batch RL algorithm that utilizes VQC as function approximators within the discrete batch-constraint deep Q-learning (BCQ) algorithm. Additionally, we introduce a novel data re-uploading scheme by cyclically shifting the order of input variables in the data encoding layers. We evaluate the efficiency of our algorithm on the OpenAI CartPole environment and compare its performance to the classical neural network-based discrete BCQ.

References in corpus (22)

• Quantum Circuit Learning
• On the Convergence of Adam and Beyond
• Error mitigation for short-depth quantum circuits
• Evaluating analytic gradients on quantum hardware
• Circuit-centric quantum classifiers
• Scalable Quantum Simulation of Molecular Energies
• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
• The effect of data encoding on the expressive power of variational quantum machine learning models
• Data re-uploading for a universal quantum classifier
• Generalization in quantum machine learning from few training data
• Quantum reinforcement learning
• Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning
• Benchmarking Batch Deep Reinforcement Learning Algorithms
• Performance comparison of optimization methods on variational quantum algorithms
• A Comparison of Various Classical Optimizers for a Variational Quantum Linear Solver
• Quantum Machine Learning for Particle Physics using a Variational Quantum Classifier
• Quantum computing of the ${}^6$Li nucleus via ordered unitary coupled clusters
• A Survey on Quantum Reinforcement Learning
• Uncovering Instabilities in Variational-Quantum Deep Q-Networks
• An Empirical Comparison of Optimizers for Quantum Machine Learning with SPSA-based Gradients
• Hybrid quantum-classical classifier based on tensor network and variational quantum circuit
• A scale-dependent notion of effective dimension

Cited by in corpus (2)

• Guided-SPSA: Simultaneous Perturbation Stochastic Approximation assisted by the Parameter Shift Rule
• Model-based Offline Quantum Reinforcement Learning