Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Search

Kayhan Behdin, Qingquan Song, Sriram Vasudevan +17

Large Language Models (LLMs) have demonstrated impressive quality when applied to predictive tasks such as relevance ranking and semantic search. However, deployment of such LLMs r…