Variational Quantum Circuit Design for Quantum Reinforcement Learning on Continuous Environments
arXiv:2312.13798 · doi:10.5220/0012353100003636
Abstract
Quantum Reinforcement Learning (QRL) emerged as a branch of reinforcement learning (RL) that uses quantum submodules in the architecture of the algorithm. One branch of QRL focuses on the replacement of neural networks (NN) by variational quantum circuits (VQC) as function approximators. Initial works have shown promising results on classical environments with discrete action spaces, but many of the proposed architectural design choices of the VQC lack a detailed investigation. Hence, in this work we investigate the impact of VQC design choices such as angle embedding, encoding block architecture and postprocessesing on the training capabilities of QRL agents. We show that VQC design greatly influences training performance and heuristically derive enhancements for the analyzed components. Additionally, we show how to design a QRL agent in order to solve classical environments with continuous action spaces and benchmark our agents against classical feed-forward NNs.
References in corpus (15)
- Trust Region Policy Optimization
- Barren plateaus in quantum neural network training landscapes
- A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
- The effect of data encoding on the expressive power of variational quantum machine learning models
- OpenAI Gym
- Generalization in quantum machine learning from few training data
- A Review of Deep Reinforcement Learning for Smart Building Energy Management
- A generative modeling approach for benchmarking and training shallow quantum circuits
- Beyond Barren Plateaus: Quantum Variational Algorithms Are Swamped With Traps
- Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning
- Theory of overparametrization in quantum neural networks
- Generalization in Quantum Machine Learning: a Quantum Information Perspective
- Variational Quantum Soft Actor-Critic
- Quantum Policy Gradient Algorithm with Optimized Action Decoding
- Variational Quantum Soft Actor-Critic for Robotic Arm Control