Model-aware reinforcement learning for high-performance Bayesian experimental design in quantum metrology
arXiv:2312.16985 · doi:10.22331/q-2024-12-10-1555
Abstract
Quantum sensors offer control flexibility during estimation by allowing manipulation by the experimenter across various parameters. For each sensing platform, pinpointing the optimal controls to enhance the sensor's precision remains a challenging task. While an analytical solution might be out of reach, machine learning offers a promising avenue for many systems of interest, especially given the capabilities of contemporary hardware. We have introduced a versatile procedure capable of optimizing a wide range of problems in quantum metrology, estimation, and hypothesis testing by combining model-aware reinforcement learning (RL) with Bayesian estimation based on particle filtering. To achieve this, we had to address the challenge of incorporating the many non-differentiable steps of the estimation in the training process, such as measurements and the resampling of the particle filter. Model-aware RL is a gradient-based method, where the derivatives of the sensor's precision are obtained through automatic differentiation (AD) in the simulation of the experiment. Our approach is suitable for optimizing both non-adaptive and adaptive strategies, using neural networks or other agents. We provide an implementation of this technique in the form of a Python library called qsensoropt, alongside several pre-made applications for relevant physical platforms, namely NV centers, photonic circuits, and optical cavities. This library will be released soon on PyPI. Leveraging our method, we've achieved results for many examples that surpass the current state-of-the-art in experimental design. In addition to Bayesian estimation, leveraging model-aware RL, it is also possible to find optimal controls for the minimization of the Cramér-Rao bound, based on Fisher information.
45 pages, 10 figures
References in corpus (62)
- Sensitivity Optimization for NV-Diamond Magnetometry
- Reinforcement Learning in Different Phases of Quantum Control
- Experimental Quantum Hamiltonian Learning
- Reinforcement Learning with Neural Networks for Quantum Feedback
- Hamiltonian Learning and Certification Using Quantum Resources
- Robust Online Hamiltonian Learning
- Optimal metrology with programmable quantum sensors
- Optimized quantum sensing with a single electron spin using real-time adaptive measurements
- Experimental neural network enhanced quantum tomography
- Artificial Intelligence and Machine Learning for Quantum Technologies
- Learning Quantum Systems
- Introduction to Quantum Optimal Control for Quantum Sensing with Nitrogen-Vacancy Centers in Diamond
- Quantum parameter estimation with optimal control
- When does reinforcement learning stand out in quantum control? A comparative study on state preparation
- A variational toolbox for quantum multi-parameter estimation
- Multi-qubit gates and Schrödinger cat states in an optical clock
- Quantum Variational Optimization of Ramsey Interferometry and Atomic Clocks
- Generalizable control for quantum parameter estimation through reinforcement learning
- Displacement receiver for phase-shift-keyed coherent states
- Distinguishing between optical coherent states with imperfect detection
- Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements
- Learning models of quantum systems from experiments
- How to best sample a periodic probability distribution, or on the accuracy of Hamiltonian finding strategies
- Implementation of projective measurements with linear optics and continuous photon counting
- Calibration of quantum sensors by neural networks
- QuanEstimation: An open-source toolkit for quantum parameter estimation
- Revisiting the Dolinar Receiver through Multiple-Copy State Discrimination Theory
- Variational principle for optimal quantum controls in quantum metrology
- Learning feedback control strategies for quantum metrology
- Deep reinforcement learning for quantum multiparameter estimation
- Bayesian estimation for quantum sensing in the absence of single-shot detection
- Optimal and Variational Multi-Parameter Quantum Metrology and Vector Field Sensing
- Neural-Network Heuristics for Adaptive Bayesian Quantum Estimation
- Improving the dynamics of quantum sensors with reinforcement learning
- Magnetic-field-learning using a single electronic spin in diamond with one-photon-readout at room temperature
- Machine-learning-enhanced quantum sensors for accurate magnetic field imaging
- QInfer: Statistical inference software for quantum applications
- Gradient Ascent Pulse Engineering with Feedback
- Deep Reinforcement Learning for Efficient Measurement of Quantum Devices
- Adaptive tracking of a time-varying field with a quantum sensor
- Quantum learning of coherent states
- Photonic architecture for reinforcement learning
- Generalizable control for multiparameter quantum metrology
- Machine Learning and Quantum Devices
- Robust Magnetometry with Single NV Centers via Two-step Optimization
- Efficient and Robust Entanglement Generation with Deep Reinforcement Learning for Quantum Metrology
- Real-time calibration of coherent-state receivers: learning by trial and error
- Quantum Receiver Enhanced by Adaptive Learning
- Non-asymptotic Heisenberg scaling: experimental metrology for a wide resources range
- Designing optimal protocols in Bayesian quantum parameter estimation with higher-order operations
- Adaptive Hamiltonian Estimation Using Bayesian Experimental Design
- Neural-network-based parameter estimation for quantum detection
- Resource-efficient adaptive Bayesian tracking of magnetic fields with a quantum sensor
- Atomic clock locking with Bayesian quantum parameter estimation: scheme and experiment
- Adaptive Discrimination Scheme for Quantum Pulse Position Modulation Signals
- Photonic quantum metrology with variational quantum optical non-linearities
- Real-time adaptive estimation of decoherence timescales for a single qubit
- An agnostic-Dolinar receiver for coherent states classification
- Real-time frequency estimation of a qubit without single-shot-readout
- Super-Resolution Imaging with Multiparameter Quantum Metrology in Passive Remote Sensing
- Applications of model-aware reinforcement learning in Bayesian quantum metrology
- Framework for Learning and Control in the Classical and Quantum Domains