FIVES: Feature Interaction Via Edge Search for Large-Scale Tabular Data

Abstract

High-order interactive features capture the correlation between different columns and thus are promising to enhance various learning tasks on ubiquitous tabular data. To automate the generation of interactive features, existing works either explicitly traverse the feature space or implicitly express the interactions via intermediate activations of some designed models. These two kinds of methods show that there is essentially a trade-off between feature interpretability and search efficiency. To possess both of their merits, we propose a novel method named Feature Interaction Via Edge Search (FIVES), which formulates the task of interactive feature generation as searching for edges on the defined feature graph. Specifically, we first present our theoretical evidence that motivates us to search for useful interactive features with increasing order. Then we instantiate this search strategy by optimizing both a dedicated graph neural network (GNN) and the adjacency tensor associated with the defined feature graph. In this way, the proposed FIVES method simplifies the time-consuming traversal as a typical training course of GNN and enables explicit feature generation according to the learned adjacency tensor. Experimental results on both benchmark and real-world datasets show the advantages of FIVES over several state-of-the-art methods. Moreover, the interactive features identified by FIVES are deployed on the recommender system of Taobao, a worldwide leading e-commerce platform. Results of an online A/B testing further verify the effectiveness of the proposed method FIVES, and we further provide FIVES as AI utilities for the customers of Alibaba Cloud.

Accepted by KDD-21

References in corpus (9)

• Categorical Reparameterization with Gumbel-Softmax
• DARTS: Differentiable Architecture Search
• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction
• BayesNAS: A Bayesian Approach for Neural Architecture Search
• ASAP: Architecture Search, Anneal and Prune
• Network On Network for Tabular Data Classification in Real-world Applications
• Feature Interaction Interpretability: A Case for Explaining Ad-Recommendation Systems via Neural Interaction Detection
• DNN2LR: Interpretation-inspired Feature Crossing for Real-world Tabular Data