Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language models

Abstract

To retrieve and compare scientific data of simulations and experiments in materials science, data needs to be easily accessible and machine readable to qualify and quantify various materials science phenomena. The recent progress in open science leverages the accessibility to data. However, a majority of information is encoded within scientific documents limiting the capability of finding suitable literature as well as material properties. This manuscript showcases an automated workflow, which unravels the encoded information from scientific literature to a machine readable data structure of texts, figures, tables, equations and meta-data, using natural language processing and language as well as vision transformer models to generate a machine-readable database. The machine-readable database can be enriched with local data, as e.g. unpublished or private material data, leading to knowledge synthesis. The study shows that such an automated workflow accelerates information retrieval, proximate context detection and material property extraction from multi-modal input data exemplarily shown for the research field of microstructural analyses of face-centered cubic single crystals. Ultimately, a Retrieval-Augmented Generation (RAG) based Large Language Model (LLM) enables a fast and efficient question answering chat bot.

References in corpus (20)

• LLaMA: Open and Efficient Foundation Language Models
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Recent Advances and Applications of Deep Learning Methods in Materials Science
• Visual Instruction Tuning
• MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
• Opportunities and Challenges for Machine Learning in Materials Science
• Extracting Accurate Materials Data from Research Papers with Conversational Language Models and Prompt Engineering
• 14 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathon
• Large Language Models Are State-of-the-Art Evaluators of Translation Quality
• Materials science in the era of large language models: a perspective
• Accelerating Material Design with the Generative Toolkit for Scientific Discovery
• A rule-free workflow for the automated generation of databases from scientific literature
• Mining experimental data from Materials Science literature with Large Language Models: an evaluation study
• Nougat: Neural Optical Understanding for Academic Documents
• From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design
• Identification of dislocation reaction kinetics in complex dislocation networks for continuum modeling using data-driven methods
• Microstructure evolution of compressed micropillars investigated by in situ HR-EBSD analysis and dislocation density simulations
• Characterization of Lomer junctions based on the Lomer arm length distribution in dislocation networks
• A graph database for feature characterization of dislocation networks
• Exploring large language models for microstructure evolution in materials