AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction

Abstract

Learning feature interactions is crucial for click-through rate (CTR) prediction in recommender systems. In most existing deep learning models, feature interactions are either manually designed or simply enumerated. However, enumerating all feature interactions brings large memory and computation cost. Even worse, useless interactions may introduce noise and complicate the training process. In this work, we propose a two-stage algorithm called Automatic Feature Interaction Selection (AutoFIS). AutoFIS can automatically identify important feature interactions for factorization models with computational cost just equivalent to training the target model to convergence. In the \emph{search stage}, instead of searching over a discrete set of candidate feature interactions, we relax the choices to be continuous by introducing the architecture parameters. By implementing a regularized optimizer over the architecture parameters, the model can automatically identify and remove the redundant feature interactions during the training process of the model. In the \emph{re-train stage}, we keep the architecture parameters serving as an attention unit to further boost the performance. Offline experiments on three large-scale datasets (two public benchmarks, one private) demonstrate that AutoFIS can significantly improve various FM based models. AutoFIS has been deployed onto the training platform of Huawei App Store recommendation service, where a 10-day online A/B test demonstrated that AutoFIS improved the DeepFM model by 20.3\% and 20.1\% in terms of CTR and CVR respectively.

arXiv commentKDD 2020 ADS track oral accepted

References in corpus (7)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• DARTS: Differentiable Architecture Search
• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• Wide & Deep Learning for Recommender Systems
• Product-based Neural Networks for User Response Prediction
• A generalization of regularized dual averaging and its dynamics
• Hierarchical Neural Architecture Search via Operator Clustering

Cited by in corpus (19)

• DARTS+: Improved Differentiable Architecture Search with Early Stopping
• OptEmbed: Learning Optimal Embedding Table for Click-through Rate Prediction
• DVR: Micro-Video Recommendation Optimizing Watch-Time-Gain under Duration Bias
• Detecting Arbitrary Order Beneficial Feature Interactions for Recommender Systems
• Reformulating CTR Prediction: Learning Invariant Feature Interactions for Recommendation
• Memory-efficient Embedding for Recommendations
• A General Method For Automatic Discovery of Powerful Interactions In Click-Through Rate Prediction
• MvFS: Multi-view Feature Selection for Recommender System
• Directional Pruning of Deep Neural Networks
• COLD: Towards the Next Generation of Pre-Ranking System
• Learning and Optimization of Implicit Negative Feedback for Industrial Short-video Recommender System
• Hierarchical Neural Architecture Search via Operator Clustering
• VEGA: Towards an End-to-End Configurable AutoML Pipeline
• AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System
• AutoLoss: Automated Loss Function Search in Recommendations
• i-Razor: A Differentiable Neural Input Razor for Feature Selection and Dimension Search in DNN-Based Recommender Systems
• DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
• DNN2LR: Interpretation-inspired Feature Crossing for Real-world Tabular Data
• NEON: Living Needs Prediction System in Meituan