Dynamic Grammar-Compressed Self-Index in $\hat{I}´$-Optimal Space

Abstract

A compressed self-index stores a string in compressed form while supporting locate queries without decompression. For highly repetitive strings, such as those arising in web crawls, versioned documents, and genomic collections, static self-indexes can match the $\hat{I}´$-optimal lower bound of $\hat{I}\text{c◯}(\hat{I}´\log (n\log \ddot{I}ƒ/(\hat{I}´\log n))\log n)$ bits up to constant factors, where $n$ is the string length, $\ddot{I}ƒ$ is the alphabet size, and $\hat{I}´$ is the substring complexity. Their dynamic counterparts, however, remain scarce: every existing dynamic self-index either fails to attain $\hat{I}´$-optimal space, pays $\hat{I}\text{c◯}(\log n)$ time per reported occurrence for locate queries, or has an update time that grows with the maximum value in the longest common prefix (LCP) array of the text. We present the dynamic RR-index, a dynamic grammar-compressed self-index built on the restricted recompression run-length straight-line program (RLSLP). To our knowledge, it is the first dynamic self-index to attain $\hat{I}´$-optimal space. The index uses $O(\hat{I}´\log (n\log \ddot{I}ƒ/(\hat{I}´\log n))\log n)$ bits in expectation, answers locate queries in expected $O(m+\log m{\log }^{2}n+occ(\log n/\log \log n))$ time, where $m$ is the pattern length and $occ$ is the number of occurrences, and supports insertion of a length-$m^{\prime }$ string and deletion of a length-$m^{\prime }$ substring in expected amortized $O(m^{\prime }{\log }^{2}n+{\log }^{3}n)$ time, with no dependence on the maximum LCP value. On eleven highly repetitive corpora, including a $37$ GB Wikipedia dump and a $59$ GB human-chromosome collection, the dynamic RR-index is up to $77\times$ faster than the dynamic r-index for updates and up to $11\times$ faster than other dynamic indexes for locate queries.

Revised in response to the reviewers' comments