HierarchicalKV: A GPU Hash Table with Cache Semantics for Continuous Online Embedding Storage

Haidong Rong, Jiashu Yao, Matthias Langer +12

Traditional GPU hash tables preserve every inserted key -- a dictionary assumption that wastes scarce High Bandwidth Memory (HBM) when embedding tables routinely exceed single-GPU…