Active Inverse Reward Design

Sören Mindermann, Rohin Shah, Adam Gleave +1

Designers of AI agents often iterate on the reward function in a trial-and-error process until they get the desired behavior, but this only guarantees good behavior in the training…