Universal Multi-Party Poisoning Attacks

Abstract

In this work, we demonstrate universal multi-party poisoning attacks that adapt and apply to any multi-party learning process with arbitrary interaction pattern between the parties. More generally, we introduce and study $(k,p)$-poisoning attacks in which an adversary controls $k\in [m]$ of the parties, and for each corrupted party $P_i$, the adversary submits some poisoned data ${\mathcal{T}}_i^{\prime }$ on behalf of $P_i$ that is still ``$(1-p)$-close'' to the correct data ${\mathcal{T}}_i$ (e.g., $1-p$ fraction of ${\mathcal{T}}_i^{\prime }$ is still honestly generated). We prove that for any ``bad'' property $B$ of the final trained hypothesis $h$ (e.g., $h$ failing on a particular test example or having ``large'' risk) that has an arbitrarily small constant probability of happening without the attack, there always is a $(k,p)$-poisoning attack that increases the probability of $B$ from $\mu$ to by ${\mu }^{1-p\cdot k/m}=\mu +\Omega (p\cdot k/m)$. Our attack only uses clean labels, and it is online. More generally, we prove that for any bounded function $f(x_1,\dots ,x_n)\in [0,1]$ defined over an $n$-step random process $\mathbf{X}=(x_1,\dots ,x_n)$, an adversary who can override each of the $n$ blocks with even dependent probability $p$ can increase the expected output by at least $\Omega (p\cdot \mathrm{Var}[f(\mathbf{x})])$.

Cited by in corpus (6)

• Model-Targeted Poisoning Attacks with Provable Convergence
• On the Sample Complexity of Adversarial Multi-Source PAC Learning
• Robust learning under clean-label attack
• Learning and Certification under Instance-targeted Poisoning
• Widen The Backdoor To Let More Attackers In
• A Separation Result Between Data-oblivious and Data-aware Poisoning Attacks