Image and Data Mining in Reticular Chemistry Using GPT-4V

Abstract

The integration of artificial intelligence into scientific research has reached a new pinnacle with GPT-4V, a large language model featuring enhanced vision capabilities, accessible through ChatGPT or an API. This study demonstrates the remarkable ability of GPT-4V to navigate and obtain complex data for metal-organic frameworks, especially from graphical sources. Our approach involved an automated process of converting 346 scholarly articles into 6240 images, which represents a benchmark dataset in this task, followed by deploying GPT-4V to categorize and analyze these images using natural language prompts. This methodology enabled GPT-4V to accurately identify and interpret key plots integral to MOF characterization, such as nitrogen isotherms, PXRD patterns, and TGA curves, among others, with accuracy and recall above 93%. The model's proficiency in extracting critical information from these plots not only underscores its capability in data mining but also highlights its potential in aiding the creation of comprehensive digital databases for reticular chemistry. In addition, the extracted nitrogen isotherm data from the selected literature allowed for a comparison between theoretical and experimental porosity values for over 200 compounds, highlighting certain discrepancies and underscoring the importance of integrating computational and experimental data. This work highlights the potential of AI in accelerating scientific discovery and innovation, bridging the gap between computational tools and experimental research, and paving the way for more efficient, inclusive, and comprehensive scientific inquiry.

36 pages, 24 figures

References in corpus (12)

• Llama 2: Open Foundation and Fine-Tuned Chat Models
• ChatGPT Chemistry Assistant for Text Mining and Prediction of MOF Synthesis
• 14 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathon
• Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models
• Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
• The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
• PaLM 2 Technical Report
• A GPT-4 Reticular Chemist for Guiding MOF Discovery
• The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4
• DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
• Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V
• Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks

Cited by in corpus (4)

• Materials science in the era of large language models: a perspective
• El Agente: An Autonomous Agent for Quantum Chemistry
• The Rise of Generative AI for Metal-Organic Framework Design and Synthesis
• Developing an AI Course for Synthetic Chemistry Students