Koa-decision: Fast and Consistent Structured Decision Making with Generative LLMs

Shenghong Dai, Shiva Kumar Pentyala, Yingchi Liu +6

Industry applications often demand low-latency classification, yet current large language model (LLM) approaches remain poorly suited for latency-critical applications. Existing pr…