Improving Long-Context Retrieval with Multi-Prefix Embedding

Zhenglin Yu, Xueguang Ma, Shengyao Zhuang +4

Long-context retrieval exposes a tension: single-vector embeddings lose fine-grained detail, while token-level multi-vector methods incur prohibitive storage. We propose Multi-Pref…