Product-based Neural Networks for User Response Prediction

Abstract

Predicting user responses, such as clicks and conversions, is of great importance and has found its usage in many Web applications including recommender systems, web search and online advertising. The data in those applications is mostly categorical and contains multiple fields; a typical representation is to transform it into a high-dimensional sparse binary feature representation via one-hot encoding. Facing with the extreme sparsity, traditional models may limit their capacity of mining shallow patterns from the data, i.e. low-order feature combinations. Deep models like deep neural networks, on the other hand, cannot be directly applied for the high-dimensional input because of the huge feature space. In this paper, we propose a Product-based Neural Networks (PNN) with an embedding layer to learn a distributed representation of the categorical data, a product layer to capture interactive patterns between inter-field categories, and further fully connected layers to explore high-order feature interactions. Our experimental results on two large-scale real-world ad click datasets demonstrate that PNNs consistently outperform the state-of-the-art models on various metrics.

arXiv comment6 pages, 5 figures, ICDM2016

References in corpus (3)

• Distributed Representations of Words and Phrases and their Compositionality
• Real-Time Bidding Benchmarking with iPinYou Dataset
• Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction

Cited by in corpus (18)

• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• Learning Tree-based Deep Model for Recommender Systems
• Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networks
• Deep Interest Network for Click-Through Rate Prediction
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction
• DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction
• Mixture of Virtual-Kernel Experts for Multi-Objective User Profile Modeling
• Entire Space Multi-Task Modeling via Post-Click Behavior Decomposition for Conversion Rate Prediction
• Pairwise Interactive Graph Attention Network for Context-Aware Recommendation
• Augmenting Recurrent Neural Networks with High-Order User-Contextual Preference for Session-Based Recommendation
• Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction
• Learning to Recommend via Meta Parameter Partition
• Denoising User-aware Memory Network for Recommendation
• DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
• Simple Modifications to Improve Tabular Neural Networks
• AdnFM: An Attentive DenseNet based Factorization Machine for CTR Prediction
• A Recommendation System to Enhance Midwives' Capacities in Low-Income Countries
• SHORING: Design Provable Conditional High-Order Interaction Network via Symbolic Testing