Deep Learning for Automated Experimentation in Scanning Transmission Electron Microscopy

Abstract

Machine learning (ML) has become critical for post-acquisition data analysis in (scanning) transmission electron microscopy, (S)TEM, imaging and spectroscopy. An emerging trend is the transition to real-time analysis and closed-loop microscope operation. The effective use of ML in electron microscopy now requires the development of strategies for microscopy-centered experiment workflow design and optimization. Here, we discuss the associated challenges with the transition to active ML, including sequential data analysis and out-of-distribution drift effects, the requirements for the edge operation, local and cloud data storage, and theory in the loop operations. Specifically, we discuss the relative contributions of human scientists and ML agents in the ideation, orchestration, and execution of experimental workflows and the need to develop universal hyper languages that can apply across multiple platforms. These considerations will collectively inform the operationalization of ML in next-generation experimentation.

Review Article

References in corpus (6)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations
• MPFit: A robust method for fitting atomic resolution images with multiple Gaussian peaks
• A roadmap for edge computing enabled automated multidimensional transmission electron microscopy
• Data Mining Graphene: Correlative Analysis of Structure and Electronic Degrees of Freedom in Graphenic Monolayers with Defects

Cited by in corpus (7)

• SAM-I-Am: Semantic Boosting for Zero-shot Atomic-Scale Electron Micrograph Segmentation
• Direct visualization of defect-controlled diffusion in van der Waals gaps
• Artificial Intelligence-Assisted Workflow for Transmission Electron Microscopy: From Data Analysis Automation to Materials Knowledge Unveiling
• Mind the Gap: Bridging the Divide Between AI Aspirations and the Reality of Autonomous Characterization
• ARIM-mdx Data System: Towards a Nationwide Data Platform for Materials Science
• Mic-hackathon 2024: Hackathon on Machine Learning for Electron and Scanning Probe Microscopy
• Revealing the Evolution of Order in Materials Microstructures Using Multi-Modal Computer Vision