Domain-specific ChatBots for Science using Embeddings
arXiv:2306.10067 · doi:10.1039/D3DD00112A
Abstract
Large language models (LLMs) have emerged as powerful machine-learning systems capable of handling a myriad of tasks. Tuned versions of these systems have been turned into chatbots that can respond to user queries on a vast diversity of topics, providing informative and creative replies. However, their application to physical science research remains limited owing to their incomplete knowledge in these areas, contrasted with the needs of rigor and sourcing in science domains. Here, we demonstrate how existing methods and software tools can be easily combined to yield a domain-specific chatbot. The system ingests scientific documents in existing formats, and uses text embedding lookup to provide the LLM with domain-specific contextual information when composing its reply. We similarly demonstrate that existing image embedding methods can be used for search and retrieval across publication figures. These results confirm that LLMs are already suitable for use by physical scientists in accelerating their research efforts.
arXiv comment14 pages, 6 figures
References in corpus (38)
- LoRA: Low-Rank Adaptation of Large Language Models
- Hierarchical Text-Conditional Image Generation with CLIP Latents
- On the Opportunities and Risks of Foundation Models
- Longformer: The Long-Document Transformer
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
- Large Language Models are Zero-Shot Reasoners
- Emergent Abilities of Large Language Models
- High-Resolution Image Synthesis with Latent Diffusion Models
- ReAct: Synergizing Reasoning and Acting in Language Models
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- QLoRA: Efficient Finetuning of Quantized LLMs
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Toolformer: Language Models Can Teach Themselves to Use Tools
- Fine-Tuning Language Models from Human Preferences
- Measuring Massive Multitask Language Understanding
- Reflexion: Language Agents with Verbal Reinforcement Learning
- The Creativity of Text-to-Image Generation
- Galactica: A Large Language Model for Science
- HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
- ChatGPT is not all you need. A State of the Art Review of large Generative AI models
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- PAL: Program-aided Language Models
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
- What learning algorithm is in-context learning? Investigations with linear models
- Emergent autonomous scientific research capabilities of large language models
- Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
- Progress measures for grokking via mechanistic interpretability
- LongNet: Scaling Transformers to 1,000,000,000 Tokens
- Let's Verify Step by Step
- Large Language Models as Tool Makers
- Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting
- ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
- Focused Transformer: Contrastive Training for Context Scaling
- Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models
- TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs
- Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling