Autonomous robotic nanofabrication with reinforcement learning

Abstract

The ability to handle single molecules as effectively as macroscopic building-blocks would enable the construction of complex supramolecular structures inaccessible to self-assembly. The fundamental challenges obstructing this goal are the uncontrolled variability and poor observability of atomic-scale conformations. Here, we present a strategy to work around both obstacles, and demonstrate autonomous robotic nanofabrication by manipulating single molecules. Our approach employs reinforcement learning (RL), which finds solution strategies even in the face of large uncertainty and sparse feedback. We demonstrate the potential of our RL approach by removing molecules autonomously with a scanning probe microscope from a supramolecular structure -- an exemplary task of subtractive manufacturing at the nanoscale. Our RL agent reaches an excellent performance, enabling us to automate a task which previously had to be performed by a human. We anticipate that our work opens the way towards autonomous agents for the robotic construction of functional supramolecular structures with speed, precision and perseverance beyond our current capabilities.

arXiv comment3 figures

References in corpus (5)

• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Gaussian Processes for Data-Efficient Learning in Robotics and Control
• Superlubricity of Graphene Nanoribbons on Gold Surfaces
• Scanning Quantum Dot Microscopy
• Scanning Probe State Recognition With Multi-Class Neural Network Ensembles

Cited by in corpus (12)

• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• SchNetPack 2.0: A neural network toolbox for atomistic machine learning
• BIGDML: Towards Exact Machine Learning Force Fields for Materials
• Precise atom manipulation through deep reinforcement learning
• Autonomous Investigations over WS${}_2$ and Au{111} with Scanning Probe Microscopy
• Machine Learning and Data-Driven Methods in Computational Surface and Interface Science
• Free energy along drug-protein binding pathways interactively sampled in virtual reality
• Precise large-scale chemical transformations on surfaces: deep learning meets scanning probe microscopy with interpretability
• Integration of Scanning Probe Microscope with High-Performance Computing: fixed-policy and reward-driven workflows implementation
• Mic-hackathon 2024: Hackathon on Machine Learning for Electron and Scanning Probe Microscopy
• Electrostatic Discovery Atomic Force Microscopy
• AI-Guided Molecular Simulations in VR: Exploring Strategies for Imitation Learning in Hyperdimensional Molecular Systems