Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction

Abstract

CTR prediction in real-world business is a difficult machine learning problem with large scale nonlinear sparse data. In this paper, we introduce an industrial strength solution with model named Large Scale Piece-wise Linear Model (LS-PLM). We formulate the learning problem with $L_1$ and $L_{2,1}$ regularizers, leading to a non-convex and non-smooth optimization problem. Then, we propose a novel algorithm to solve it efficiently, based on directional derivatives and quasi-Newton method. In addition, we design a distributed system which can run on hundreds of machines parallel and provides us with the industrial scalability. LS-PLM model can capture nonlinear patterns from massive sparse data, saving us from heavy feature engineering jobs. Since 2012, LS-PLM has become the main CTR prediction model in Alibaba's online display advertising system, serving hundreds of millions users every day.

Cited by in corpus (15)

• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• Learning Tree-based Deep Model for Recommender Systems
• Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction
• Deep Interest Network for Click-Through Rate Prediction
• Scenario-Adaptive and Self-Supervised Model for Multi-Scenario Personalized Recommendation
• A Deep Prediction Network for Understanding Advertiser Intent and Satisfaction
• COLD: Towards the Next Generation of Pre-Ranking System
• Matching Algorithms: Fundamentals, Applications and Challenges
• Image Matters: Visually modeling user behaviors using Advanced Model Server
• Improving Native Ads CTR Prediction by Large Scale Event Embedding and Recurrent Networks
• Res-embedding for Deep Learning Based Click-Through Rate Prediction Modeling
• Spatial-Temporal Deep Intention Destination Networks for Online Travel Planning
• CIA-Towards a Unified Marketing Optimization Framework for e-Commerce Sponsored Search
• Decision Machines: Congruent Decision Trees
• Deep Personalized Re-targeting