Deep Learning for Automated Experimentation in Scanning Transmission Electron Microscopy

Abstract

Machine learning (ML) has become critical for post-acquisition data analysis in (scanning) transmission electron microscopy, (S)TEM, imaging and spectroscopy. An emerging trend is the transition to real-time analysis and closed-loop microscope operation. The effective use of ML in electron microscopy now requires the development of strategies for microscopy-centered experiment workflow design and optimization. Here, we discuss the associated challenges with the transition to active ML, including sequential data analysis and out-of-distribution drift effects, the requirements for the edge operation, local and cloud data storage, and theory in the loop operations. Specifically, we discuss the relative contributions of human scientists and ML agents in the ideation, orchestration, and execution of experimental workflows and the need to develop universal hyper languages that can apply across multiple platforms. These considerations will collectively inform the operationalization of ML in next-generation experimentation.

arXiv commentReview Article

References in corpus (22)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• Fast inference of deep neural networks in FPGAs for particle physics
• Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations
• Octahedral rotation-induced ferroelectricity in cation ordered perovskites
• Turning ABO${}_3$ antiferroelectrics into ferroelectrics: Design rules for practical rotation-driven ferroelectricity in double perovskites and A${}_3$B${}_2$O${}_7$ Ruddlesden-Popper compounds
• Spectroscopic imaging of single atoms within a bulk solid
• Atomic-scale compensation phenomena at polar interfaces
• Deep Learning Enabled Strain Mapping of Single-Atom Defects in 2D Transition Metal Dichalcogenides with Sub-picometer Precision
• Strain Mapping of Two-Dimensional Heterostructures with Sub-Picometer Precision
• Exploring order parameters and dynamic processes in disordered systems via variational autoencoders
• Physics discovery in nanoplasmonic systems via autonomous experiments in Scanning Transmission Electron Microscopy
• An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics
• MPFit: A robust method for fitting atomic resolution images with multiple Gaussian peaks
• Imaging Atomic-Scale Chemistry from Fused Multi-Modal Electron Microscopy
• Design of a Graphical User Interface for Few-Shot Machine Learning Classification of Electron Microscopy Data
• Direct-Writing Atom-by-Atom
• A roadmap for edge computing enabled automated multidimensional transmission electron microscopy
• Data Mining Graphene: Correlative Analysis of Structure and Electronic Degrees of Freedom in Graphenic Monolayers with Defects
• Identification and Correction of Temporal and Spatial Distortions in Scanning Transmission Electron Microscopy
• Atomic Scale Measurement of Polar Entropy
• Evaluating Stage Motion for Automated Electron Microscopy

Cited by in corpus (7)

• SAM-I-Am: Semantic Boosting for Zero-shot Atomic-Scale Electron Micrograph Segmentation
• Direct visualization of defect-controlled diffusion in van der Waals gaps
• Artificial Intelligence-Assisted Workflow for Transmission Electron Microscopy: From Data Analysis Automation to Materials Knowledge Unveiling
• Mind the Gap: Bridging the Divide Between AI Aspirations and the Reality of Autonomous Characterization
• ARIM-mdx Data System: Towards a Nationwide Data Platform for Materials Science
• Mic-hackathon 2024: Hackathon on Machine Learning for Electron and Scanning Probe Microscopy
• Revealing the Evolution of Order in Materials Microstructures Using Multi-Modal Computer Vision