Parameterized Reinforcement Learning for Optical System Optimization

Abstract

Designing a multi-layer optical system with designated optical characteristics is an inverse design problem in which the resulting design is determined by several discrete and continuous parameters. In particular, we consider three design parameters to describe a multi-layer stack: Each layer's dielectric material and thickness as well as the total number of layers. Such a combination of both, discrete and continuous parameters is a challenging optimization problem that often requires a computationally expensive search for an optimal system design. Hence, most methods merely determine the optimal thicknesses of the system's layers. To incorporate layer material and the total number of layers as well, we propose a method that considers the stacking of consecutive layers as parameterized actions in a Markov decision process. We propose an exponentially transformed reward signal that eases policy optimization and adapt a recent variant of Q-learning for inverse design optimization. We demonstrate that our method outperforms human experts and a naive reinforcement learning algorithm concerning the achieved optical characteristics. Moreover, the learned Q-values contain information about the optical properties of multi-layer optical systems, thereby allowing physical interpretation or what-if analysis.

arXiv commentPresented as a poster at the workshop on machine learning for engineering modeling, simulation and design @ NeurIPS 2020

References in corpus (18)

• Continuous control with deep reinforcement learning
• Playing Atari with Deep Reinforcement Learning
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Prioritized Experience Replay
• Training deep neural networks for the inverse design of nanophotonic structures
• Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• A Generative Model for Inverse Design of Metamaterials
• Opening the Black Box of Deep Neural Networks via Information
• Intelligent Nanophotonics: Merging Photonics and Artificial Intelligence at the Nanoscale
• Multilayer optical calculations
• Efficient spectrum prediction and inverse design for plasmonic waveguide systems based on artificial neural networks
• Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space
• Pushing the limits of optical information storage using deep learning
• Reinforcement Learning with Parameterized Actions
• Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces
• Machine Learning enables Design of On-chip Integrated Silicon T-junctions with footprint of 1.2 $\mu$m x 1.2 $\mu$m
• Deep reinforcement learning for optical systems: A case study of mode-locked lasers

Cited by in corpus (4)

• TMM-Fast: A Transfer Matrix Computation Package for Multilayer Thin-Film Optimization
• Inverse design of nano-photonic wavelength demultiplexer with a deep neural network approach
• Investigation of inverse design of multilayer thin-films with conditional invertible Neural Networks
• Monte Carlo Tree Search for high precision manufacturing