Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference

Jianxing Qin, Alexander Du, Danfeng Zhang +2

LLM inference systems may vary batch composition, prompt chunking, prefill/decode execution, and KV-cache reuse, eviction, or recomputation. These optimizations should not affect s…