ADOR: A Design Exploration Framework for LLM Serving with Enhanced Latency and Throughput

Junsoo Kim, Hunjong Lee, Geonwoo Ko +4

The growing adoption of Large Language Models (LLMs) across various domains has driven the demand for efficient and scalable AI-serving solutions. Deploying LLMs requires optimizat…