Photonic architecture for reinforcement learning

Abstract

The last decade has seen an unprecedented growth in artificial intelligence and photonic technologies, both of which drive the limits of modern-day computing devices. In line with these recent developments, this work brings together the state of the art of both fields within the framework of reinforcement learning. We present the blueprint for a photonic implementation of an active learning machine incorporating contemporary algorithms such as SARSA, Q-learning, and projective simulation. We numerically investigate its performance within typical reinforcement learning environments, showing that realistic levels of experimental noise can be tolerated or even be beneficial for the learning process. Remarkably, the architecture itself enables mechanisms of abstraction and generalization, two features which are often considered key ingredients for artificial intelligence. The proposed architecture, based on single-photon evolution on a mesh of tunable beamsplitters, is simple, scalable, and a first integration in portable systems appears to be within the reach of near-term technology.

arXiv comment14 pages, 8 figures

References in corpus (19)

• Deep Learning with Coherent Nanophotonic Circuits
• Photonic quantum information processing: a review
• Training of photonic neural networks through in situ backpropagation
• Self-Configuring Universal Linear Optical Component
• All Optical Neural Network with Nonlinear Activation Functions
• Reinforcement Learning in Different Phases of Quantum Control
• Active learning machine learns to create new quantum experiments
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Self-aligning universal beam coupler
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• AlphaStar: An Evolutionary Computation Perspective
• Coherent Transport of Quantum States by Deep Reinforcement Learning
• Direct dialling of Haar random unitary matrices
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Projective simulation with generalization
• Meta-learning within Projective Simulation
• Modelling collective motion based on the principle of agency
• Benchmarking projective simulation in navigation problems
• Robotic Playing for Hierarchical Complex Skill Learning

Cited by in corpus (11)

• Experimental quantum speed-up in reinforcement learning agents
• Machine learning for long-distance quantum communication
• Robust calibration of multiparameter sensors via machine learning at the single-photon level
• Operationally meaningful representations of physical systems in neural networks
• A hybrid classical-quantum approach to speed-up Q-learning
• Performance analysis of a hybrid agent for quantum-accessible reinforcement learning
• Applications of model-aware reinforcement learning in Bayesian quantum metrology
• Model-aware reinforcement learning for high-performance Bayesian experimental design in quantum metrology
• Conflict-free joint decision by lag and zero-lag synchronization in laser network
• Demonstration of quantum projective simulation on a single-photon-based quantum computer
• Emergence of biased errors in imperfect photonic circuits