CSR: Infinite-Horizon Real-Time Policies with Massive Cached State Representations

Robin Karlsson, Go Suzui

Deploying massive large language models (LLMs) as continuous cognitive engines for robotics is bottlenecked by the time-to-first-token (TTFT) latency required to process extensive…