NASRec: Weight Sharing Neural Architecture Search for Recommender Systems

Abstract

The rise of deep neural networks offers new opportunities in optimizing recommender systems. However, optimizing recommender systems using deep neural networks requires delicate architecture fabrication. We propose NASRec, a paradigm that trains a single supernet and efficiently produces abundant models/sub-architectures by weight sharing. To overcome the data multi-modality and architecture heterogeneity challenges in the recommendation domain, NASRec establishes a large supernet (i.e., search space) to search the full architectures. The supernet incorporates versatile choice of operators and dense connectivity to minimize human efforts for finding priors. The scale and heterogeneity in NASRec impose several challenges, such as training inefficiency, operator-imbalance, and degraded rank correlation. We tackle these challenges by proposing single-operator any-connection sampling, operator-balancing interaction modules, and post-training fine-tuning. Our crafted models, NASRecNet, show promising results on three Click-Through Rates (CTR) prediction benchmarks, indicating that NASRec outperforms both manually designed models and existing NAS methods with state-of-the-art performance. Our work is publicly available at https://github.com/facebookresearch/NasRec.

arXiv commentProceedings of the ACM Web Conference 2023 (WWW'23)

References in corpus (4)

• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
• DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
• Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction