A Tight Analysis of Hutchinson's Diagonal Estimator

Abstract

Let $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ be a matrix with diagonal $\text{diag}(\mathbf{A})$ and let $\bar{\mathbf{A}}$ be $\mathbf{A}$ with its diagonal set to all zeros. We show that Hutchinson's estimator run for $m$ iterations returns a diagonal estimate $\tilde{d}\in {\mathbb{R}}^n$ such that with probability $(1-\delta )$, $$\Vert \tilde{d}-\text{diag}(\mathbf{A}){\Vert }_2\le c\sqrt{\frac{\log (2/\delta )}{m}}\Vert \bar{\mathbf{A}}{\Vert }_F,$$ where $c$ is a fixed constant independent of all other parameters. This results improves on a recent result of [Baston and Nakatsukasa, 2022] by a $\log (n)$ factor, yielding a bound that is independent of the matrix dimension $n$.

To appear in SIAM Symposium on Simplicity in Algorithms (SOSA23)