Why AI Safety Requires Uncertainty, Incomplete Preferences, and Non-Archimedean Utilities

Alessio Benavoli, Alessandro Facchini, Marco Zaffalon

How can we ensure that AI systems are aligned with human values and remain safe? We can study this problem through the frameworks of the AI assistance and the AI shutdown games. Th…