Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction

Abstract

Predicting user responses, such as click-through rate and conversion rate, are critical in many web applications including web search, personalised recommendation, and online advertising. Different from continuous raw features that we usually found in the image and audio domains, the input features in web space are always of multi-field and are mostly discrete and categorical while their dependencies are little known. Major user response prediction models have to either limit themselves to linear models or require manually building up high-order combination features. The former loses the ability of exploring feature interactions, while the latter results in a heavy computation in the large feature space. To tackle the issue, we propose two novel models using deep neural networks (DNNs) to automatically learn effective patterns from categorical feature interactions and make predictions of users' ad clicks. To get our DNNs efficiently work, we propose to leverage three feature transformation methods, i.e., factorisation machines (FMs), restricted Boltzmann machines (RBMs) and denoising auto-encoders (DAEs). This paper presents the structure of our models and their efficient training algorithms. The large-scale experiments with real-world data demonstrate that our methods work better than major state-of-the-art models.

References in corpus (2)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Natural Language Processing (almost) from Scratch

Cited by in corpus (12)

• DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
• Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction
• Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction
• Product-based Neural Networks for User Response Prediction
• RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms
• DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction
• PBODL : Parallel Bayesian Online Deep Learning for Click-Through Rate Prediction in Tencent Advertising System
• ICE: Information Credibility Evaluation on Social Media via Representation Learning
• Muddling Label Regularization: Deep Learning for Tabular Datasets
• Retrieval & Interaction Machine for Tabular Data Prediction
• Deep CTR Prediction in Display Advertising
• Dense Moving Fog for Intelligent IoT: Key Challenges and Opportunities