"We Need Structured Output": Towards User-centered Constraints on Large Language Model Output

Abstract

Large language models can produce creative and diverse responses. However, to integrate them into current developer workflows, it is essential to constrain their outputs to follow specific formats or standards. In this work, we surveyed 51 experienced industry professionals to understand the range of scenarios and motivations driving the need for output constraints from a user-centered perspective. We identified 134 concrete use cases for constraints at two levels: low-level, which ensures the output adhere to a structured format and an appropriate length, and high-level, which requires the output to follow semantic and stylistic guidelines without hallucination. Critically, applying output constraints could not only streamline the currently repetitive process of developing, testing, and integrating LLM prompts for developers, but also enhance the user experience of LLM-powered features and applications. We conclude with a discussion on user preferences and needs towards articulating intended constraints for LLMs, alongside an initial design for a constraint prototyping tool.

References in corpus (17)

• Training language models to follow instructions with human feedback
• PaLM: Scaling Language Modeling with Pathways
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
• Holistic Evaluation of Language Models
• "What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models
• Prompting Is Programming: A Query Language for Large Language Models
• Wigglite: Low-cost Information Collection and Triage
• TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
• Instruction-Following Evaluation for Large Language Models
• Large Language Models for Software Engineering: Survey and Open Problems
• Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making
• Evaluating Large Language Models at Evaluating Instruction Following
• Building Your Own Product Copilot: Challenges, Opportunities, and Needs
• Visual Studio Code in Introductory Computer Science Course: An Experience Report
• ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principles
• Controlled Decoding from Language Models
• LLM Comparator: Visual Analytics for Side-by-Side Evaluation of Large Language Models

Cited by in corpus (5)

• VeriPlan: Integrating Formal Verification and LLMs into End-User Planning
• Semantic Integrity Constraints: Declarative Guardrails for AI-Augmented Data Processing Systems
• Gensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoning
• Guided Decoding and Its Critical Role in Retrieval-Augmented Generation
• OOPrompt: Reifying Intents into Structured Artifacts for Modular and Iterative Prompting