Materials science in the era of large language models: a perspective

Abstract

Large Language Models (LLMs) have garnered considerable interest due to their impressive natural language capabilities, which in conjunction with various emergent properties make them versatile tools in workflows ranging from complex code generation to heuristic finding for combinatorial problems. In this paper we offer a perspective on their applicability to materials science research, arguing their ability to handle ambiguous requirements across a range of tasks and disciplines mean they could be a powerful tool to aid researchers. We qualitatively examine basic LLM theory, connecting it to relevant properties and techniques in the literature before providing two case studies that demonstrate their use in task automation and knowledge extraction at-scale. At their current stage of development, we argue LLMs should be viewed less as oracles of novel insight, and more as tireless workers that can accelerate and unify exploration across domains. It is our hope that this paper can familiarise material science researchers with the concepts needed to leverage these tools in their own research.

References in corpus (56)

• Training language models to follow instructions with human feedback
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• LoRA: Low-Rank Adaptation of Large Language Models
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
• Gemini: A Family of Highly Capable Multimodal Models
• A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
• Retrieval-Augmented Generation for Large Language Models: A Survey
• Visual Instruction Tuning
• Segment Anything
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Toolformer: Language Models Can Teach Themselves to Use Tools
• PaLM-E: An Embodied Multimodal Language Model
• Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• Reflexion: Language Agents with Verbal Reinforcement Learning
• 14 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathon
• Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models
• The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only
• ChemBERTa-2: Towards Chemical Foundation Models
• Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
• Larger language models do in-context learning differently
• Large Language Models as Optimizers
• MatterGen: a generative model for inorganic materials design
• Image and Data Mining in Reticular Chemistry Using GPT-4V
• The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4
• MemGPT: Towards LLMs as Operating Systems
• Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4
• Eureka: Human-Level Reward Design via Coding Large Language Models
• Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions
• Fine-Tuned Language Models Generate Stable Inorganic Materials as Text
• LLM-Prop: Predicting Physical And Electronic Properties Of Crystalline Solids From Their Text Descriptions
• A Review of Machine Learning Applications for the Proton Magnetic Resonance Spectroscopy Workflow
• Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
• Large Language Models Can Self-Improve
• Large Language Models as Tool Makers
• Explainability for Large Language Models: A Survey
• Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering
• Crystal Structure Generation with Autoregressive Large Language Modeling
• Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey
• LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models
• GPT-MolBERTa: GPT Molecular Features Language Model for molecular property prediction
• DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents
• ViperGPT: Visual Inference via Python Execution for Reasoning
• World Model on Million-Length Video And Language With Blockwise RingAttention
• OLMo: Accelerating the Science of Language Models
• Mitigating the Alignment Tax of RLHF
• GPT Can Solve Mathematical Problems Without a Calculator
• Accurate Prediction of Experimental Band Gaps from Large Language Model-Based Data Extraction
• MolXPT: Wrapping Molecules with Text for Generative Pre-training
• Language-Driven Representation Learning for Robotics
• Vector Search with OpenAI Embeddings: Lucene Is All You Need
• Housekeep: Tidying Virtual Households using Commonsense Reasoning
• Adversarial Quantum Machine Learning: An Information-Theoretic Generalization Analysis
• OpenAi's GPT4 as coding assistant

Cited by in corpus (6)

• AI-driven inverse design of materials: Past, present and future
• Materials Informatics: Emergence To Autonomous Discovery In The Age Of AI
• Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language models
• Dara: Automated multiple-hypothesis phase identification and refinement from powder X-ray diffraction
• aLLoyM: A large language model for alloy phase diagram prediction
• Large Language Models as AI Agents for Digital Atoms and Molecules: Catalyzing a New Era in Computational Biophysics