Groups generated by derangements

Abstract

We examine the subgroup $D(G)$ of a transitive permutation group $G$ which is generated by the derangements in $G$. Our main results bound the index of this subgroup: we conjecture that, if $G$ has degree $n$ and is not a Frobenius group, then $|G:D(G)|⩽\sqrt{n}-1$; we prove this except when $G$ is a primitive affine group. For affine groups, we translate our conjecture into an equivalent form regarding $|H:R(H)|$, where $H$ is a linear group on a finite vector space and $R(H)$ is the subgroup of $H$ generated by elements having eigenvalue~$1$. If $G$ is a Frobenius group, then $D(G)$ is the Frobenius kernel, and so $G/D(G)$ is isomorphic to a Frobenius complement. We give some examples where $D(G)\ne G$, and examine the group-theoretic structure of $G/D(G)$; in particular, we construct groups $G$ in which $G/D(G)$ is not a Frobenius complement.