4 papers
When KV Meets Embeddings: Dynamic GPU Memory Allocation for Accelerating Generative Recommender Serving
Wenjun Yu, Shuguang Han, Amelie Chi Zhou
Generative Recommender (GR) inference places embedding hot caches (EMB) and KV caches in direct competition for limited GPU HBM: allocating more memory to one improves its efficien…
Equity vs. Equality: Optimizing Ranking Fairness for Tailored Provider Needs
Yiteng Tu, Weihang Su, Shuguang Han +2
Ranking plays a central role in connecting users and providers in Information Retrieval (IR) systems, making provider-side fairness an important challenge. While recent research ha…
AIF: Asynchronous Inference Framework for Cost-Effective Pre-Ranking
Zhi Kou, Xiang-Rong Sheng, Shuguang Han +5
In industrial recommendation systems, pre-ranking models based on deep neural networks (DNNs) commonly adopt a sequential execution framework: feature fetching and model forward co…
Calibration-compatible Listwise Distillation of Privileged Features for CTR Prediction
Xiaoqiang Gui, Yueyao Cheng, Xiang-Rong Sheng +6
In machine learning systems, privileged features refer to the features that are available during offline training but inaccessible for online serving. Previous studies have recogni…