16 citations · 23 across the 8 of their papers we have counts for
9 papers
Reward Models Inherit Value Biases from Pretraining
Brian Christian, Jessica A. F. Thompson, Elle Michelle Yang +4
Reward models (RMs) are central to aligning large language models (LLMs) with human values but have received less attention than pretrained and post-trained LLMs themselves. Becaus…
Neural steering vectors reveal dose and exposure-dependent impacts of human-AI relationships
Hannah Rose Kirk, Henry Davidson, Ed Saunders +4
Humans are increasingly forming parasocial relationships with AI systems, and modern AI shows an increasing tendency to display social and relationship-seeking behaviour. However,…
People readily follow personal advice from AI but it does not improve their well-being
Lennart Luettgau, Vanessa Cheung, Magda Dubois +8
People increasingly seek personal advice from large language models (LLMs), yet whether humans follow their advice, and its consequences for their well-being, remains unknown. In a…
Conversational AI increases political knowledge as effectively as self-directed internet search
Lennart Luettgau, Hannah Rose Kirk, Kobi Hackenburg +6
Conversational AI systems are increasingly being used in place of traditional search engines to help users complete information-seeking tasks. This has raised concerns in the polit…
Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language
Christopher Summerfield, Lennart Luettgau, Magda Dubois +9
We examine recent research that asks whether current AI systems may be developing a capacity for "scheming" (covertly and strategically pursuing misaligned goals). We compare curre…
Reward Model Interpretability via Optimal and Pessimal Tokens
Brian Christian, Hannah Rose Kirk, Jessica A. F. Thompson +2
Reward modeling has emerged as a crucial component in aligning large language models with human values. Significant attention has focused on using reward models as a means for fine…
Beyond the Binary: Capturing Diverse Preferences With Reward Regularization
Vishakh Padmakumar, Chuanyang Jin, Hannah Rose Kirk +1
Large language models (LLMs) are increasingly deployed via public-facing interfaces to interact with millions of users, each with diverse preferences. Despite this, preference tuni…
The Future of Open Human Feedback
Shachar Don-Yehiya, Ben Burtenshaw, Ramon Fernandez Astudillo +17
Human feedback on conversations with language language models (LLMs) is central to how these systems learn about the world, improve their capabilities, and are steered toward desir…
The AI Community Building the Future? A Quantitative Analysis of Development Activity on Hugging Face Hub
Cailean Osborne, Jennifer Ding, Hannah Rose Kirk
Open model developers have emerged as key actors in the political economy of artificial intelligence (AI), but we still have a limited understanding of collaborative practices in t…