On the Largest Product-free Subsets of the Alternating Groups

Abstract

A subset $A$ of a group $G$ is called product-free if there is no solution to $a=bc$ with $a,b,c$ all in $A$. It is easy to see that the largest product-free subset of the symmetric group $S_n$ is obtained by taking the set of all odd permutations, i.e. $S_n\setminus A_n$, where $A_n$ is the alternating group. By contrast, it is a long-standing open problem to find the largest product-free subset of $A_n$. We solve this problem for large $n$, showing that the maximum size is achieved by the previously conjectured extremal examples, namely families of the form $\{\pi |\pi (x)\in I,\pi (I)\cap I=\varnothing \}$ and their inverses. Moreover, we show that the maximum size is only achieved by these extremal examples, and we have stability: any product-free subset of $A_n$ of nearly maximum size is structurally close to an extremal example. Our proof uses a combination of tools from Combinatorics and Non-abelian Fourier Analysis, including a crucial new ingredient exploiting some recent theory developed by Filmus, Kindler, Liftshitz and Minzer for global hypercontractivity on the symmetric group.