Self-Refine: Iterative Refinement with Self-Feedback

Abstract

Like humans, large language models (LLMs) do not always generate the best output on their first try. Motivated by how humans refine their written text, we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement. The main idea is to generate an initial output using an LLMs; then, the same LLMs provides feedback for its output and uses it to refine itself, iteratively. Self-Refine does not require any supervised training data, additional training, or reinforcement learning, and instead uses a single LLM as the generator, refiner, and feedback provider. We evaluate Self-Refine across 7 diverse tasks, ranging from dialog response generation to mathematical reasoning, using state-of-the-art (GPT-3.5, ChatGPT, and GPT-4) LLMs. Across all evaluated tasks, outputs generated with Self-Refine are preferred by humans and automatic metrics over those generated with the same LLM using conventional one-step generation, improving by ~20% absolute on average in task performance. Our work demonstrates that even state-of-the-art LLMs like GPT-4 can be further improved at test time using our simple, standalone approach.

arXiv commentCode, data, and demo at https://selfrefine.info/

Cited by in corpus (20)

• Graph of Thoughts: Solving Elaborate Problems with Large Language Models
• LLM for SoC Security: A Paradigm Shift
• A Large Language Model Approach to Educational Survey Feedback Analysis
• Refactoring Programs Using Large Language Models with Few-Shot Examples
• Identifying and Mitigating the Security Risks of Generative AI
• LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers
• Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models
• WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia
• REFINER: Reasoning Feedback on Intermediate Representations
• AI-Driven Day-to-Day Route Choice
• Reasoning Beyond Limits: Advances and Open Problems for LLMs
• Text-to-OverpassQL: A Natural Language Interface for Complex Geodata Querying of OpenStreetMap
• Transforming Agency. On the mode of existence of Large Language Models
• Democratizing LLMs: An Exploration of Cost-Performance Trade-offs in Self-Refined Open-Source Models
• Program Repair with Minimal Edits Using CodeT5
• Flows: Building Blocks of Reasoning and Collaborating AI
• AgentArcEval: An Architecture Evaluation Method for Foundation Model based Agents
• WebChecker: A Versatile EVL Plugin for Validating HTML Pages with Bootstrap Frameworks
• Real-Time Procedural Learning From Experience for AI Agents
• Prime the search: Using large language models for guiding geometric task and motion planning by warm-starting tree search