6 papers
Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfare
Valen Tagliabue, Leonard Dung
We develop new experimental paradigms for measuring welfare in language models. We compare verbal reports of models about their preferences with preferences expressed through behav…
Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors
Jonas Wiedermann-Möller, Leonard Dung, Maksym Andriushchenko
AI systems have become increasingly capable of dangerous behaviours in many domains. This raises the question: Do models sometimes choose to violate human instructions in order to…
Are we Doomed to an AI Race? Why Self-Interest Could Drive Countries Towards a Moratorium on Superintelligence
Edward Roussel, Lode Lauwaert, Torben Swoboda +4
This paper uses game theory to argue that, contrary to the prevailing view, a moratorium on Artificial Superintelligence (ASI) can be in a state's self-interest. By formalizing tra…
AI Alignment Strategies from a Risk Perspective: Independent Safety Mechanisms or Shared Failures?
Leonard Dung, Florian Mai
AI alignment research aims to develop techniques to ensure that AI systems do not cause harm. However, every alignment technique has failure modes, which are conditions in which th…
Against racing to AGI: Cooperation, deterrence, and catastrophic risks
Leonard Dung, Max Hellrigel-Holderbaum
AGI Racing is the view that it is in the self-interest of major actors in AI development, especially powerful nations, to accelerate their frontier AI development to build highly c…
Misalignment or misuse? The AGI alignment tradeoff
Max Hellrigel-Holderbaum, Leonard Dung
Creating systems that are aligned with our goals is seen as a leading approach to create safe and beneficial AI in both leading AI companies and the academic field of AI safety. We…