Watermarked Game Solving via Perturbed Regret Minimization

Abstract

Many real-world interactions among self-interested parties can be modeled by game theory, and the rapid advancements in AI have raised concerns about the possible misuse---accidental or deliberate---of superhuman or human-level game-playing agents by bad actors. While AI watermarking has mainly been applied to LLM-generated texts, a recent line of work proposes developing watermarking techniques for agents in game-theoretic settings. However, existing watermarking techniques for game-theoretic agents are not readily applicable due to their limited scope or capabilities---they are tailored to perfect-information games and are thus inapplicable to richer game types. We propose a new approach to watermarking game-playing agents, which a) can be applied to imperfect-information settings; b) is directly integrated into the learning process itself; and c) incurs only a bounded cost in exploitability. For this purpose, we introduce perturbed regret minimization, which adds perturbations to the utilities prior to observation so as to encourage the learning algorithm to embed the watermark. Our experiments show that the watermark incurs only a small exploitability cost and can be detected within just a couple of hours of gameplay at human speed.