ELEC: Efficient Large Language Model-Empowered Click-Through Rate Prediction

Abstract

Click-through rate (CTR) prediction plays an important role in online advertising systems. On the one hand, traditional CTR prediction models capture the collaborative signals in tabular data via feature interaction modeling, but they lose semantics in text. On the other hand, Large Language Models (LLMs) excel in understanding the context and meaning behind text, but they face challenges in capturing collaborative signals and they have long inference latency. In this paper, we aim to leverage the benefits of both types of models and pursue collaboration, semantics and efficiency. We present ELEC, which is an Efficient LLM-Empowered CTR prediction framework. We first adapt an LLM for the CTR prediction task. In order to leverage the ability of the LLM but simultaneously keep efficiency, we utilize the pseudo-siamese network which contains a gain network and a vanilla network. We inject the high-level representation vector generated by the LLM into a collaborative CTR model to form the gain network such that it can take advantage of both tabular modeling and textual modeling. However, its reliance on the LLM limits its efficiency. We then distill the knowledge from the gain network to the vanilla network on both the score level and the representation level, such that the vanilla network takes only tabular data as input, but can still generate comparable performance as the gain network. Our approach is model-agnostic. It allows for the integration with various existing LLMs and collaborative CTR models. Experiments on real-world datasets demonstrate the effectiveness and efficiency of ELEC for CTR prediction.

SIGIR 2025

References in corpus (9)

• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
• DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
• TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation
• Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising
• User Behavior Retrieval for Click-Through Rate Prediction
• EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Prediction
• Deep Spatio-Temporal Neural Networks for Click-Through Rate Prediction
• Reformulating CTR Prediction: Learning Invariant Feature Interactions for Recommendation