Randomly Permuted Orthogonal Products and Fast Dimension Reduction

Abstract

We study the effect of random signed permutations on products of orthogonal matrices and their applications to fast dimension reduction. Let $A,B\in {\mathbb{R}}^{d\times d}$ be orthogonal matrices and let $\Sigma \in {\mathbb{R}}^{d\times d}$ be a uniformly random signed permutation matrix. We analyze the random orthogonal matrix \[ U=A ΣB, \] and show that, under mild assumptions on the size of the entries of $A$ and $B$, \[ \max_{i,j=1,\ldots,d} |U_{ij}| =O\left ( \sqrt{\frac{\log d}{d}}\right ) \] with high probability. As an application, we show that ORA, an analogue of the Kac walk in which every update is a $\pi /4$ rotation, reaches the same maximal entry scale after $O(d\log d)$ updates. This resolves a question of Jain et al. and improves the running time of their construction. We also show that parallel ORA reaches this scale after $O(\log d)$ rounds. We then study the random embedding \[ Φ = \sqrt{\frac{d}{m}}\, P_I U D_{ξ'}, \] where $P_I$ restricts to $m$ coordinates and ${\xi }^{\prime }$ is an independent Rademacher vector. We identify two parameters controlling norm preservation and show that, throughout the corresponding admissible range, $\Phi$ achieves optimal embedding dimension $m\asymp {\epsilon }^{-2}\log (N)$. Finally, we extend the result to structured infinite models, including sparse vectors, low-rank matrices, and finite unions of subspaces.

35 pages, 0 figures