Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators

Abstract

Large language models that exhibit instruction-following behaviour represent one of the biggest recent upheavals in conversational interfaces, a trend in large part fuelled by the release of OpenAI's ChatGPT, a proprietary large language model for text generation fine-tuned through reinforcement learning from human feedback (LLM+RLHF). We review the risks of relying on proprietary software and survey the first crop of open-source projects of comparable architecture and functionality. The main contribution of this paper is to show that openness is differentiated, and to offer scientific documentation of degrees of openness in this fast-moving field. We evaluate projects in terms of openness of code, training data, model weights, RLHF data, licensing, scientific documentation, and access methods. We find that while there is a fast-growing list of projects billing themselves as 'open source', many inherit undocumented data of dubious legality, few share the all-important instruction-tuning (a key site where human annotation labour is involved), and careful scientific documentation is exceedingly rare. Degrees of openness are relevant to fairness and accountability at all points, from data collection and curation to model architecture, and from training and fine-tuning to release and deployment.

References in corpus (18)

• Training language models to follow instructions with human feedback
• LLaMA: Open and Efficient Foundation Language Models
• Model Cards for Model Reporting
• The importance of transparency and reproducibility in artificial intelligence research
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Fine-Tuning Language Models from Human Preferences
• Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model
• Multimodal datasets: misogyny, pornography, and malignant stereotypes
• Augmented Language Models: a Survey
• Self-Instruct: Aligning Language Models with Self-Generated Instructions
• Interactive Model Cards: A Human-Centered Approach to Model Documentation
• The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
• Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset
• Crosslingual Generalization through Multitask Finetuning
• Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards
• Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data
• Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning
• EleutherAI: Going Beyond "Open Science" to "Science in the Open"

Cited by in corpus (7)

• A Survey on Large Language Models for Code Generation
• The Moral Machine Experiment on Large Language Models
• The AI Community Building the Future? A Quantitative Analysis of Development Activity on Hugging Face Hub
• Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings
• Mapping the individual, social, and biospheric impacts of Foundation Models
• HyperPIE: Hyperparameter Information Extraction from Scientific Publications
• On the Workflows and Smells of Leaderboard Operations (LBOps): An Exploratory Study of Foundation Model Leaderboards