Domain-specific ChatBots for Science using Embeddings
arXiv:2306.10067 · doi:10.1039/D3DD00112A
Abstract
Large language models (LLMs) have emerged as powerful machine-learning systems capable of handling a myriad of tasks. Tuned versions of these systems have been turned into chatbots that can respond to user queries on a vast diversity of topics, providing informative and creative replies. However, their application to physical science research remains limited owing to their incomplete knowledge in these areas, contrasted with the needs of rigor and sourcing in science domains. Here, we demonstrate how existing methods and software tools can be easily combined to yield a domain-specific chatbot. The system ingests scientific documents in existing formats, and uses text embedding lookup to provide the LLM with domain-specific contextual information when composing its reply. We similarly demonstrate that existing image embedding methods can be used for search and retrieval across publication figures. These results confirm that LLMs are already suitable for use by physical scientists in accelerating their research efforts.
14 pages, 6 figures
References in corpus (17)
- LoRA: Low-Rank Adaptation of Large Language Models
- Hierarchical Text-Conditional Image Generation with CLIP Latents
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
- High-Resolution Image Synthesis with Latent Diffusion Models
- QLoRA: Efficient Finetuning of Quantized LLMs
- Toolformer: Language Models Can Teach Themselves to Use Tools
- Galactica: A Large Language Model for Science
- ChatGPT is not all you need. A State of the Art Review of large Generative AI models
- Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
- Emergent autonomous scientific research capabilities of large language models
- LongNet: Scaling Transformers to 1,000,000,000 Tokens
- Let's Verify Step by Step
- Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting
- ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
- Focused Transformer: Contrastive Training for Context Scaling
- Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models
- TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs