Executing as You Generate: Hiding Execution Latency in LLM Code Interpreters

Zhensu Sun, Zhihao Lin, Zhi Chen +4

Current LLM systems are increasingly equipped with a code interpreter that executes generated code to obtain results. This works serially: the model first generates the complete co…