Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
arXiv:2309.00779 · doi:10.1609/aaai.v38i18.29970
Abstract
Human values are crucial to human decision-making. Value pluralism is the view that multiple correct values may be held in tension with one another (e.g., when considering lying to a friend to protect their feelings, how does one balance honesty with friendship?). As statistical learners, AI systems fit to averages by default, washing out these potentially irreducible value conflicts. To improve AI systems to better reflect value pluralism, the first-order challenge is to explore the extent to which AI systems can model pluralistic human values, rights, and duties as well as their interaction. We introduce ValuePrism, a large-scale dataset of 218k values, rights, and duties connected to 31k human-written situations. ValuePrism's contextualized values are generated by GPT-4 and deemed high-quality by human annotators 91% of the time. We conduct a large-scale study with annotators across diverse social and demographic backgrounds to try to understand whose values are represented. With ValuePrism, we build Kaleido, an open, light-weight, and structured language-based multi-task model that generates, explains, and assesses the relevance and valence (i.e., support or oppose) of human values, rights, and duties within a specific context. Humans prefer the sets of values output by our system over the teacher GPT-4, finding them more accurate and with broader coverage. In addition, we demonstrate that Kaleido can help explain variability in human decision-making by outputting contrasting values. Finally, we show that Kaleido's representations transfer to other philosophical frameworks and datasets, confirming the benefit of an explicit, modular, and interpretable approach to value pluralism. We hope that our work will serve as a step to making more explicit the implicit values behind human decision-making and to steering AI systems to make decisions that are more in accordance with them.
Proceedings of the AAAI Conference on Artificial Intelligence, 38
References in corpus (24)
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- Scaling Instruction-Finetuned Language Models
- ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks
- Out of One, Many: Using Language Models to Simulate Human Samples
- Constitutional AI: Harmlessness from AI Feedback
- Jury Learning: Integrating Dissenting Voices into Machine Learning Models
- Whose Opinions Do Language Models Reflect?
- Aligning AI With Shared Human Values
- Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
- Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
- Quark: Controllable Text Generation with Reinforced Unlearning
- Towards Measuring the Representation of Subjective Global Opinions in Language Models
- How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources
- Fine-Grained Human Feedback Gives Better Rewards for Language Model Training
- A Human Rights-Based Approach to Responsible AI
- Evaluating the Moral Beliefs Encoded in LLMs
- Towards Coding Social Science Datasets with Language Models
- The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation
- DICES Dataset: Diversity in Conversational AI Evaluation for Safety
- ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations
- Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing
- Learning Ambiguity from Crowd Sequential Annotations
- COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements
- Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation