A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
arXiv:2302.11382
Abstract
Prompt engineering is an increasingly important skill set needed to converse effectively with large language models (LLMs), such as ChatGPT. Prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output. Prompts are also a form of programming that can customize the outputs and interactions with an LLM. This paper describes a catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs. Prompt patterns are a knowledge transfer method analogous to software patterns since they provide reusable solutions to common problems faced in a particular context, i.e., output generation and interaction when working with LLMs. This paper provides the following contributions to research on prompt engineering that apply LLMs to automate software development tasks. First, it provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains. Second, it presents a catalog of patterns that have been applied successfully to improve the outputs of LLM conversations. Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.
Cited by in corpus (43)
- Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
- ChatGPT: Jack of all trades, master of none
- Unleashing the potential of prompt engineering for large language models
- Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction
- A GPT-4 Reticular Chemist for Guiding MOF Discovery
- Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics
- Prompt engineering paradigms for medical applications: scoping review and recommendations for better practices
- Large Language Models Streamline Automated Machine Learning for Clinical Studies
- MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots
- How understanding large language models can inform the use of ChatGPT in physics education
- Materials science in the era of large language models: a perspective
- "HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social media
- Assessing the Ability of ChatGPT to Screen Articles for Systematic Reviews
- A Large Language Model Approach to Educational Survey Feedback Analysis
- Educational impacts of generative artificial intelligence on learning and performance of engineering students in China
- Retrieving Supporting Evidence for Generative Question Answering
- The FormAI Dataset: Generative AI in Software Security Through the Lens of Formal Verification
- "It's not like Jarvis, but it's pretty close!" -- Examining ChatGPT's Usage among Undergraduate Students in Computer Science
- Understanding EFL Student Idea Generation Strategies for Creative Writing with NLG Tools
- From Intention To Implementation: Automating Biomedical Research via LLMs
- CoverUp: Effective High Coverage Test Generation for Python
- Innovating Computer Programming Pedagogy: The AI-Lab Framework for Generative AI Adoption
- A Reliable Knowledge Processing Framework for Combustion Science using Foundation Models
- Will Code Remain a Relevant User Interface for End-User Programming with Generative AI Models?
- Welcome Your New AI Teammate: On Safety Analysis by Leashing Large Language Models
- Isolating Compiler Bugs by Generating Effective Witness Programs with Large Language Models
- CliniDigest: A Case Study in Large Language Model Based Large-Scale Summarization of Clinical Trial Descriptions
- Evaluating Privacy Questions From Stack Overflow: Can ChatGPT Compete?
- AI-generated stories favour stability over change: homogeneity and cultural stereotyping in narratives generated by gpt-4o-mini
- Large Language Models to the Rescue: Reducing the Complexity in Scientific Workflow Development Using ChatGPT
- Vision-Language and Large Language Model Performance in Gastroenterology: GPT, Claude, Llama, Phi, Mistral, Gemma, and Quantized Models
- Programming-by-Demonstration for Long-Horizon Robot Tasks
- "Cold, Calculated, and Condescending": How AI Identifies and Explains Ableism Compared to Disabled People
- LLMs and Stack Overflow Discussions: Reliability, Impact, and Challenges
- Understanding Learner-LLM Chatbot Interactions and the Impact of Prompting Guidelines
- Topic Classification of Case Law Using a Large Language Model and a New Taxonomy for UK Law: AI Insights into Summary Judgment
- Large corpora and large language models: a replicable method for automating grammatical annotation
- Agnostic Visual Recommendation Systems: Open Challenges and Future Directions
- "The Data Says Otherwise"-Towards Automated Fact-checking and Communication of Data Claims
- Reducing Large Language Model Safety Risks in Women's Health using Semantic Entropy
- Visual Analysis of LLM-based Entity Resolution from Scientific Papers
- ChatGPT for President! Presupposed content in politicians versus GPT-generated texts
- Using Artificial Populations to Study Psychological Phenomena in Neural Models