Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations

Abstract

Recent advances in scanning transmission electron and scanning probe microscopies have opened exciting opportunities in probing the materials structural parameters and various functional properties in real space with angstrom-level precision. This progress has been accompanied by an exponential increase in the size and quality of datasets produced by microscopic and spectroscopic experimental techniques. These developments necessitate adequate methods for extracting relevant physical and chemical information from the large datasets, for which a priori information on the structures of various atomic configurations and lattice defects is limited or absent. Here we demonstrate an application of deep neural networks to extract information from atomically resolved images including location of the atomic species and type of defects. We develop a 'weakly-supervised' approach that uses information on the coordinates of all atomic species in the image, extracted via a deep neural network, to identify a rich variety of defects that are not part of an initial training set. We further apply our approach to interpret complex atomic and defect transformation, including switching between different coordination of silicon dopants in graphene as a function of time, formation of peculiar silicon dimer with mixed 3-fold and 4-fold coordination, and the motion of molecular 'rotor'. This deep learning based approach resembles logic of a human operator, but can be scaled leading to significant shift in the way of extracting and analyzing information from raw experimental data.

References in corpus (7)

• Machine learning phases of matter
• Searching for Exotic Particles in High-Energy Physics with Deep Learning
• Learning phase transitions by confusion
• Observing Atomic Collapse Resonances in Artificial Nuclei on Graphene
• Single atom manipulation and control in a scanning transmission electron microscope
• Computational insights and the observation of SiC nanograin assembly: towards 2D silicon carbide
• Mitigating E-beam-induced Hydrocarbon Deposition on Graphene for Atomic-Scale Scanning Transmission Electron Microscopy Studies

Cited by in corpus (72)

• Deep learning in nano-photonics: inverse design and beyond
• Opportunities and Challenges for Machine Learning in Materials Science
• Deep learning meets nanophotonics: A generalized accurate predictor for near fields and far fields of arbitrary 3D nanostructures
• A deep learning approach to identify local structures in atomic-resolution transmission electron microscopy images
• Deep Learning Analysis of Defect and Phase Evolution During Electron Beam Induced Transformations in WS2
• Deep Learning Enabled Strain Mapping of Single-Atom Defects in 2D Transition Metal Dichalcogenides with Sub-picometer Precision
• Machine learning in nuclear materials research
• Electron-Beam Manipulation of Silicon Dopants in Graphene
• Deep Learning for Automated Experimentation in Scanning Transmission Electron Microscopy
• Review: Deep Learning in Electron Microscopy
• Artificial Intelligent Atomic Force Microscope Enabled by Machine Learning
• A Deep Learning Based Automatic Defect Analysis Framework for In-situ TEM Ion Irradiations
• Exploring order parameters and dynamic processes in disordered systems via variational autoencoders
• Machine Learning Pipeline for Segmentation and Defect Identification from High Resolution Transmission Electron Microscopy Data
• Doping transition-metal atoms in graphene for atomic-scale tailoring of electronic, magnetic, and quantum topological properties
• Multi defect detection and analysis of electron microscopy images with deep learning
• Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images
• Perspective: Challenges and Transformative Opportunities in Superconductor Vortex Physics
• A deep learning approach for determining the chiral indices of carbon nanotubes from high-resolution transmission electron microscopy images
• STEM image analysis based on deep learning: identification of vacancy defects and polymorphs of $Mo{S}_2$
• Drawing Phase Diagrams of Random Quantum Systems by Deep Learning the Wave Functions
• An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics
• Partial Scanning Transmission Electron Microscopy with Deep Learning
• Machine learning assisted phase and size-controlled synthesis of iron oxides
• Autonomous scanning probe microscopy with hypothesis learning: Exploring the physics of domain switching in ferroelectric materials
• Quantifying the dynamics of protein self-organization using deep learning analysis of atomic force microscopy data
• Exploring particle dynamics during self-organization processes via rotationally invariant latent representations
• Atomic mechanisms for the Si atom dynamics in graphene: chemical transformations at the edge and in the bulk
• Deep Learning-Guided Surface Characterization for Autonomous Hydrogen Lithography
• Designing Materials Acceleration Platforms for Heterogeneous CO2 Photo(thermal)catalysis
• Tracking the Picoscale Spatial Motion of Atomic Columns During Dynamic Structural Change
• The role of temperature on defect diffusion and nanoscale patterning in graphene
• Phyllosilicates as earth-abundant layered materials for electronics and optoelectronics: Prospects and challenges in their ultrathin limit
• Near-real-time diagnosis of electron optical phase aberrations in scanning transmission electron microscopy using an artificial neural network
• Charting the low-loss region in Electron Energy Loss Spectroscopy with machine learning
• Revealing the chemical bonding in adatoms arrays via machine learning of 3D scanning tunneling spectroscopy data
• Machine learning aided atomic structure identification of interfacial ionic hydrates from atomic force microscopy images
• Auto-resolving the atomic structure at van der Waals interfaces using a generative model
• Reconstructing the exit wave in high-resolution transmission electron microscopy using machine learning
• Deep learning for nano-photonic materials -- The solution to everything!?
• Towards Augmented Microscopy with Reinforcement Learning-Enhanced Workflows
• Learning Metal Microstructural Heterogeneity through Spatial Mapping of Diffraction Latent Space Features
• Deep-Learning-Aided Extraction of Optical Constants in Scanning Near-Field Optical Microscopy
• Creation and Microscopic Origins of Single-Photon Emitters in Transition Metal Dichalcogenides and Hexagonal Boron Nitride
• SIM-STEM Lab: Incorporating Compressed Sensing Theory for Fast STEM Simulation
• Deep learning polarization distributions in ferroelectrics from STEM data: with and without atom finding
• Benchmark tests of atom segmentation deep learning models with a consistent dataset
• Evaluating Stage Motion for Automated Electron Microscopy
• Understanding Important Features of Deep Learning Models for Transmission Electron Microscopy Image Segmentation
• Integration of Scanning Probe Microscope with High-Performance Computing: fixed-policy and reward-driven workflows implementation
• Unsupervised learning of ferroic variants from atomically resolved STEM images
• Defect complexes in CrSBr revealed through electron microscopy and deep learning
• Artificial Intelligence-Assisted Workflow for Transmission Electron Microscopy: From Data Analysis Automation to Materials Knowledge Unveiling
• Deep analytics of atomically-resolved images: manifest and latent features
• Robust multi-scale multi-feature deep learning for atomic and defect identification in Scanning Tunneling Microscopy on H-Si(100) 2x1 surface
• Defect contrast with 4D-STEM: Understanding crystalline order with virtual detectors and beam modification
• Performance, Successes and Limitations of Deep Learning Semantic Segmentation of Multiple Defects in Transmission Electron Micrographs
• Dynamic Atomic Column Detection in Transmission Electron Microscopy Videos via Ridge Estimation
• Enhancing atomic-resolution in electron microscopy: A frequency-domain deep learning denoiser
• Robust recognition and exploratory analysis of crystal structures via Bayesian deep learning
• Fast and interpretable classification of small X-ray diffraction datasets using data augmentation and deep neural networks
• Autonomous detection of molecular configurations in microscopic images based on deep convolutional neural network
• Off-the-shelf deep learning is not enough: parsimony, Bayes and causality
• Bridging microscopy with molecular dynamics and quantum simulations: An AtomAI based pipeline
• Counting Molecules: Python based scheme for automated enumeration and categorization of molecules in scanning tunneling microscopy images
• Hexatic Phase in Covalent Two-Dimensional Silver Iodide
• Identification of Crystal Symmetry from Noisy Diffraction Patterns by A Shape Analysis and Deep Learning
• Application of deep neural networks for computing the renormalization group flow of the two-dimensional phi^4 field theory
• TEMImageNet Training Library and AtomSegNet Deep-Learning Models for High-Precision Atom Segmentation, Localization, Denoising, and Super-Resolution Processing of Atomic-Resolution Images
• Training artificial neural networks for precision orientation and strain mapping using 4D electron diffraction datasets
• $3D$ Crystal Image Analysis based on Fast Synchrosqueezed Transforms
• Physical discovery in representation learning via conditioning on prior knowledge: applications for ferroelectric domain dynamics