Deep learning in nano-photonics: inverse design and beyond

Abstract

Deep learning in the context of nano-photonics is mostly discussed in terms of its potential for inverse design of photonic devices or nanostructures. Many of the recent works on machine-learning inverse design are highly specific, and the drawbacks of the respective approaches are often not immediately clear. In this review we want therefore to provide a critical review on the capabilities of deep learning for inverse design and the progress which has been made so far. We classify the different deep learning-based inverse design approaches at a higher level as well as by the context of their respective applications and critically discuss their strengths and weaknesses. While a significant part of the community's attention lies on nano-photonic inverse design, deep learning has evolved as a tool for a large variety of applications. The second part of the review will focus therefore on machine learning research in nano-photonics "beyond inverse design". This spans from physics informed neural networks for tremendous acceleration of photonics simulations, over sparse data reconstruction, imaging and "knowledge discovery" to experimental applications.

Review article of 18 pages, 7 figures, 4 info-boxes

References in corpus (63)

• An Introduction to Variational Autoencoders
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• An overview of deep learning in medical imaging focusing on MRI
• One pixel attack for fooling deep neural networks
• Outlook for inverse design in nanophotonics
• Integrated Photonic Quantum Technologies
• An introduction to quantum machine learning
• Training deep neural networks for the inverse design of nanophotonic structures
• Phase recovery and holographic image reconstruction using deep learning in neural networks
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• A Generative Model for Inverse Design of Metamaterials
• Image Transmission Through an Opaque Material
• Physics-informed neural networks for inverse problems in nano-optics and metamaterials
• Deep Learning Microscopy
• Deep-STORM: super-resolution single-molecule microscopy by deep learning
• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• Mode-matching in multiresonant plasmonic nanoantennas for enhanced second harmonic generation
• Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media
• Discovering physical concepts with neural networks
• Global optimization of dielectric metasurfaces using a physics-driven neural network
• Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations
• Learning to see through multimode fibers
• Freeform Diffractive Metagrating Design Based on Generative Adversarial Networks
• Wave Physics as an Analog Recurrent Neural Network
• DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning
• Probing 10 μK stability and residual drifts in the cross-polarized dual-mode stabilization of single-crystal ultrahigh-Q optical resonators
• Active learning machine learns to create new quantum experiments
• Intelligent Nanophotonics: Merging Photonics and Artificial Intelligence at the Nanoscale
• Deep learning meets nanophotonics: A generalized accurate predictor for near fields and far fields of arbitrary 3D nanostructures
• Automated Search for new Quantum Experiments
• Convolutional neural networks automate detection for tracking of submicron scale particles in 2D and 3D
• Experimental neural network enhanced quantum tomography
• Deep learning enhanced mobile-phone microscopy
• Deep Learning and Model Predictive Control for Self-Tuning Mode-Locked Lasers
• Evolutionary optimization of optical antennas
• Evolutionary Multi-Objective Optimisation of Colour Pixels based on Dielectric Nano-Antennas
• Three dimensional waveguide-interconnects for scalable integration of photonic neural networks
• Pushing the limits of optical information storage using deep learning
• Deep Learning Reveals Underlying Physics of Light-matter Interactions in Nanophotonic Devices
• Elucidating the Behavior of Nanophotonic Structures Through Explainable Machine Learning Algorithms
• Knowledge Discovery In Nanophotonics Using Geometric Deep Learning
• Machine Learning Holography for 3D Particle Field Imaging
• Biomimetic Ultra-Broadband Perfect Absorbers Optimised with Reinforcement Learning
• Computer-inspired Quantum Experiments
• Scale-, shift- and rotation-invariant diffractive optical networks
• Plasmonic Purcell factor and coupling efficiency to surface plasmons. Implications for addressing and controlling optical nanosources
• Machine-learning techniques for fast and accurate feature localization in holograms of colloidal particles
• Iterative optimization of photonic crystal nanocavity designs by using deep neural networks
• Deep Learning Enabled Real Time Speckle Recognition and Hyperspectral Imaging using a Multimode Fiber Array
• Multi-objective and categorical global optimization of photonic structures based on ResNet generative neural networks
• A topological encoding method for data-driven photonics inverse design
• Rapid classification of quantum sources enabled by machine learning
• Partial Scanning Transmission Electron Microscopy with Deep Learning
• Fast Design of Plasmonic Metasurfaces Enabled by Deep Learning
• Probability-Density-Based Deep Learning Paradigm for the Fuzzy Design of Functional Metastructures
• Design of plasmonic directional antennas via evolutionary optimization
• Femtometer-resolved simultaneous measurement of multiple laser wavelengths in a speckle wavemeter
• Modeling and Control of a Reconfigurable Photonic Circuit using Deep Learning
• Machine learning reveals complex behaviours in optically trapped particles
• Snapshot fiber spectral imaging using speckle correlations and compressive sensing
• Optimizing illumination for precise multi-parameter estimations in coherent diffractive imaging
• Accelerating quantum optics experiments with statistical learning

Cited by in corpus (49)

• Minireview on Disordered Optical Metasurfaces
• Multiple and spectrally robust photonic magic angles in reconfigurable α-MoO3 trilayers
• Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
• A newcomer's guide to deep learning for inverse design in nano-photonics
• Inverse design of glass structure with deep graph neural networks
• Manifold Learning for Knowledge Discovery and Intelligent Inverse Design of Photonic Nanostructures: Breaking the Geometric Complexity
• Deep neural networks for the prediction of the optical properties and the free-form inverse design of metamaterials
• Deep learning enabled strategies for modelling of complex aperiodic plasmonic metasurfaces of arbitrary size
• Machine Learning of Implicit Combinatorial Rules in Mechanical Metamaterials
• Neural network design of multilayer metamaterial for temporal differentiation
• Inverse design of nano-photonic wavelength demultiplexer with a deep neural network approach
• Machine learning of phase transitions in nonlinear polariton lattices
• Deep Learning for Optical Tweezers
• Inverse design with flexible design targets via deep learning: Tailoring of electric and magnetic multipole scattering from nano-spheres
• Inverse Design of All-dielectric Metasurfaces with Bound States in the Continuum
• Deep Photonic Networks with Arbitrary and Broadband Functionality
• Data driven approaches in nanophotonics: A review of AI-enabled metadevices
• T-matrix representation of optical scattering response: Suggestion for a data format
• Machine-Learning-Assisted Photonic Device Development: A Multiscale Approach from Theory to Characterization
• Integration of Programmable Diffraction with Digital Neural Networks
• Active Physics Informed Deep Learning: Surrogate Modeling for Non Planar Wavefront Excitation of Topological Nanophotonic Devices
• Inverse-designed dielectric cloaks for entanglement generation
• Deep-Learning-Aided Extraction of Optical Constants in Scanning Near-Field Optical Microscopy
• Conditional Recurrent Neural Networks for broad applications in nonlinear optics
• Design of ultracompact broadband focusing spectrometers based on deep diffractive neural networks
• Designing the structure of an one-dimensional photonic crystal with a given spectrum of the reflection coefficient
• Freeform surface topology prediction for prescribed illumination via semi-supervised learning
• Transfer learning-assisted inverse modeling in nanophotonics based on mixture density networks
• Machine Learning-Based Optimization of Chiral Photonic Nanostructures: Evolution- and Neural Network-Based Design
• Broadband Unidirectional Visible Imaging Using Wafer-Scale Nano-Fabrication of Multi-Layer Diffractive Optical Processors
• Combining Bayesian Optimization, SVD and Machine Learning for Advanced Optical Design
• Identifying topology of leaky photonic lattices with machine learning
• Deep-learning design of graphene metasurfaces for quantum control and Dirac electron holography
• Analysis and design of transition radiation in layered uniaxial crystals using Tandem neural networks
• Fabrication uncertainty guided design optimization of a photonic crystal cavity by using Gaussian processes
• Implementing commercial inverse design tools for compact, phase-encoded, plasmonic digital logic devices
• Hybrid design of spectral splitters and concentrators of light for solar cells using iterative search and neural networks
• AI based design of 2D material integrated optical polarizers
• Magnetic and electric Purcell factor control through geometry optimization of high index dielectric nanostructures
• Quasinormal mode as a foundational framework for all electromagnetic Fano resonances
• Machine-learning-enabled characterization of individual ring resonators in integrated photonic lattices
• Fast approximate solvers for metamaterials design in electromagnetism
• Generalizing the exact multipole expansion: Density of multipole modes in complex photonic nanostructures
• Neural networks with quantum states of light
• Spectral splitting and concentration of broadband light using neural networks
• A general framework for knowledge integration in machine learning for electromagnetic scattering using quasinormal modes
• Gradient-based optimization of scatterer arrangements based on the T-matrix method
• Arbitrary Reflectionless Optical Routing via Non-Hermitian Zero-Index Networks
• Towards a universal meta-optics solver via large language models