Designing for Human-Agent Alignment: Understanding what humans want from their agents
arXiv:2404.04289 · doi:10.1145/3613905.3650948
Abstract
Our ability to build autonomous agents that leverage Generative AI continues to increase by the day. As builders and users of such agents it is unclear what parameters we need to align on before the agents start performing tasks on our behalf. To discover these parameters, we ran a qualitative empirical research study about designing agents that can negotiate during a fictional yet relatable task of selling a camera online. We found that for an agent to perform the task successfully, humans/users and agents need to align over 6 dimensions: 1) Knowledge Schema Alignment 2) Autonomy and Agency Alignment 3) Operational Alignment and Training 4) Reputational Heuristics Alignment 5) Ethics Alignment and 6) Human Engagement Alignment. These empirical findings expand previous work related to process and specification alignment and the need for values and safety in Human-AI interactions. Subsequently we discuss three design directions for designers who are imagining a world filled with Human-Agent collaborations.
Human-AI Alignment, Human-Agent Alignment, Agents, Generative AI, Large Language Models
References in corpus (5)
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions
- Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
- `It is currently hodgepodge'': Examining AI/ML Practitioners' Challenges during Co-production of Responsible AI Values
- Effects of Sensemaking Translucence on Distributed Collaborative Analysis