3 papers
cs.CL2025
Cacheback: Speculative Decoding With Nothing But Cache
Zhiyao Ma, In Gim, Lin Zhong
We present Cacheback Decoding, a training-free and model-agnostic speculative decoding method that exploits the locality in language to accelerate Large Language Model (LLM) infere…
cs.CL2025
Pie: A Programmable Serving System for Emerging LLM Applications
In Gim, Zhiyao Ma, Seung-seob Lee +1
Emerging large language model (LLM) applications involve diverse reasoning strategies and agentic workflows, straining the capabilities of existing serving systems built on a monol…
cs.PL2025
MappedTrace: Tracing Pointer Remotely with Compiler-generated Maps
Zhiyao Ma, Caihua Li, Lin Zhong
Existing precise pointer tracing methods introduce substantial runtime overhead to the program being traced and are applicable only at specific program execution points. We propose…