Mitigation of Adversarial Policy Imitation via Constrained Randomization of Policy (CRoP)

Abstract

Deep reinforcement learning (DRL) policies are vulnerable to unauthorized replication attacks, where an adversary exploits imitation learning to reproduce target policies from observed behavior. In this paper, we propose Constrained Randomization of Policy (CRoP) as a mitigation technique against such attacks. CRoP induces the execution of sub-optimal actions at random under performance loss constraints. We present a parametric analysis of CRoP, address the optimality of CRoP, and establish theoretical bounds on the adversarial budget and the expectation of loss. Furthermore, we report the experimental evaluation of CRoP in Atari environments under adversarial imitation, which demonstrate the efficacy and feasibility of our proposed method against policy replication attacks.

arXiv comment5 pages not including references; 7 figures; more figures in supplements

References in corpus (8)

• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
• Generative Adversarial Imitation Learning
• Imitation Learning as $f$-Divergence Minimization
• The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning
• Sequential Triggers for Watermarking of Deep Reinforcement Learning Policies
• Adversarial Exploitation of Policy Imitation
• Stealing Deep Reinforcement Learning Models for Fun and Profit
• PAC Bounds for Imitation and Model-based Batch Learning of Contextual Markov Decision Processes