Toward Optimal Time-Space Tradeoffs for Set Reconciliation

Abstract

Set reconciliation, where two parties each holding a large set of elements aim to identify their set difference, is a fundamental task in many areas. There are two important metrics in this problem: time (computation cost) and space (communication cost). Most previous work focuses on optimizing one metric at the expense of the other. We present XYZ-Sketch, proving that it is possible to achieve near-minimal space and $O(1)$ time updates simultaneously. Specifically, for sufficiently large $d$, XYZ-Sketch reconciles sets with only $(1+\epsilon )d$ elements for communication, while achieving $O(1)$ insertion time and $O(d\log V)$ decoding time. Here, $d$ and $V$ denote the size of the difference between two sets and the universe size, respectively. We further establish a broad fixed-support canonical model for the problem, showing that, under an open extremality conjecture, XYZ-Sketch is asymptotically optimal within this model. Experiments validate the predicted near-optimal performance of XYZ-Sketch. The source code is available at https://github.com/djwj233/XYZ-Sketch.