Publications (27)
MixLM: High-Throughput and Effective LLM Ranking via Text-Embedding Mix-Interaction
Guoyao Li, Ran He, Shusen Jing +21
Large language models (LLMs) excel at capturing semantic nuances and therefore show impressive relevance ranking performance in modern recommendation and search systems. However, t…
Semantic Search At LinkedIn
Fedor Borisyuk, Sriram Vasudevan, Muchen Wu +71
Semantic search with large language models (LLMs) enables retrieval by meaning rather than keyword overlap, but scaling it requires major inference efficiency advances. We present…
QuantEase: Optimization-based Quantization for Language Models
Kayhan Behdin, Ayan Acharya, Aman Gupta +4
With the rising popularity of Large Language Models (LLMs), there has been an increasing interest in compression techniques that enable their efficient deployment. This study focus…
Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Search
Kayhan Behdin, Qingquan Song, Sriram Vasudevan +17
Large Language Models (LLMs) have demonstrated impressive quality when applied to predictive tasks such as relevance ranking and semantic search. However, deployment of such LLMs r…
BP-Seg: A graphical model approach to unsupervised and non-contiguous text segmentation using belief propagation
Fengyi Li, Kayhan Behdin, Natesh Pillai +3
Text segmentation based on the semantic meaning of sentences is a fundamental task with broad utility in many downstream applications. In this paper, we propose a graphical model-b…
ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Models
Xiang Meng, Kayhan Behdin, Haoyue Wang +1
The impressive performance of Large Language Models (LLMs) across various natural language processing tasks comes at the cost of vast computational resources and storage requiremen…