Efficient LZ78 factorization of grammar compressed text

Abstract

We present an efficient algorithm for computing the LZ78 factorization of a text, where the text is represented as a straight line program (SLP), which is a context free grammar in the Chomsky normal form that generates a single string. Given an SLP of size $n$ representing a text $S$ of length $N$, our algorithm computes the LZ78 factorization of $T$ in $O(n\sqrt{N}+m\log N)$ time and $O(n\sqrt{N}+m)$ space, where $m$ is the number of resulting LZ78 factors. We also show how to improve the algorithm so that the $n\sqrt{N}$ term in the time and space complexities becomes either $nL$, where $L$ is the length of the longest LZ78 factor, or $(N-\alpha )$ where $\alpha \ge 0$ is a quantity which depends on the amount of redundancy that the SLP captures with respect to substrings of $S$ of a certain length. Since $m=O(N/{\log }_{\sigma }N)$ where $\sigma$ is the alphabet size, the latter is asymptotically at least as fast as a linear time algorithm which runs on the uncompressed string when $\sigma$ is constant, and can be more efficient when the text is compressible, i.e. when $m$ and $n$ are small.

SPIRE 2012

References in corpus (5)

• Tying up the loose ends in fully LZW-compressed pattern matching
• Pattern matching in Lempel-Ziv compressed strings: fast, simple, and deterministic
• On Bijective Variants of the Burrows-Wheeler Transform
• Solving Classical String Problems on Compressed Texts
• Speeding-up $q$-gram mining on grammar-based compressed texts

Cited by in corpus (1)

• RePair in Compressed Space and Time