works on
configuration-driven framework 1feature engineering 1generative retrieval 1online inference 1prompt generation 1

From the 1 of 13 linked papers with an AI index.

activity
20242026
collaborators
Showing cs.OSShow all

1 paper · 1 filter

cs.OS2026

RTP-LLM: High-Performance Alibaba LLM Inference Engine

Boyu Tan, Jiarui Guo, Zongwei Lv +26

Large Language Models (LLMs) have revolutionized AI applications, but deploying them at scale presents significant challenges. We present RTP-LLM, a high-performance inference engi…