3 papers
cs.AI2024
Towards Safe and Honest AI Agents with Neural Self-Other Overlap
Marc Carauleanu, Michael Vaiana, Judd Rosenblatt +2
As AI systems increasingly make critical decisions, deceptive AI poses a significant challenge to trust and safety. We present Self-Other Overlap (SOO) fine-tuning, a promising app…
cs.LG2024
Unexpected Benefits of Self-Modeling in Neural Systems
Vickram N. Premakumar, Michael Vaiana, Florin Pop +4
Self-models have been a topic of great interest for decades in studies of human cognition and more recently in machine learning. Yet what benefits do self-models confer? Here we sh…
cs.CL2024
Rethinking harmless refusals when fine-tuning foundation models
Florin Pop, Judd Rosenblatt, Diogo Schwerz de Lucena +1
In this paper, we investigate the degree to which fine-tuning in Large Language Models (LLMs) effectively mitigates versus merely conceals undesirable behavior. Through the lens of…