16 citations · 17 across the 3 of their papers we have counts for
3 papers
ASTRA: Accurate and Scalable ANNS-based Training of Extreme Classifiers
Sonu Mehta, Jayashree Mohan, Nagarajan Natarajan +2
`Extreme Classification'' (or XC) is the task of annotating data points (queries) with relevant labels (documents), from an extremely large set of possible labels, arising in s…
Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems
Amey Agrawal, Anmol Agarwal, Nitin Kedia +5
Serving large language models (LLMs) in production can incur substantial costs, which has prompted recent advances in inference system optimizations. Today, these systems are evalu…
SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
Amey Agrawal, Ashish Panwar, Jayashree Mohan +3
Large Language Model (LLM) inference consists of two distinct phases - prefill phase which processes the input prompt and decode phase which generates output tokens autoregressivel…