AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations

Abstract

Deep learning based recommender systems (DLRSs) often have embedding layers, which are utilized to lessen the dimensionality of categorical variables (e.g. user/item identifiers) and meaningfully transform them in the low-dimensional space. The majority of existing DLRSs empirically pre-define a fixed and unified dimension for all user/item embeddings. It is evident from recent researches that different embedding sizes are highly desired for different users/items according to their popularity. However, manually selecting embedding sizes in recommender systems can be very challenging due to the large number of users/items and the dynamic nature of their popularity. Thus, in this paper, we propose an AutoML based end-to-end framework (AutoEmb), which can enable various embedding dimensions according to the popularity in an automated and dynamic manner. To be specific, we first enhance a typical DLRS to allow various embedding dimensions; then we propose an end-to-end differentiable framework that can automatically select different embedding dimensions according to user/item popularity; finally we propose an AutoML based optimization algorithm in a streaming recommendation setting. The experimental results based on widely used benchmark datasets demonstrate the effectiveness of the AutoEmb framework.

References in corpus (13)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Neural Architecture Search with Reinforcement Learning
• Efficient Neural Architecture Search via Parameter Sharing
• Session-based Recommendations with Recurrent Neural Networks
• DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
• Large-Scale Evolution of Image Classifiers
• Neural Architecture Optimization
• SMASH: One-Shot Model Architecture Search through HyperNetworks
• SNAS: Stochastic Neural Architecture Search
• Path-Level Network Transformation for Efficient Architecture Search
• Hyperparameter optimization with approximate gradient
• Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems
• AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search

Cited by in corpus (13)

• AutoML: A Survey of the State-of-the-Art
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• Differentiable Neural Input Search for Recommender Systems
• Learnable Embedding Sizes for Recommender Systems
• Memory-efficient Embedding for Recommendations
• Deep Learning for Click-Through Rate Estimation
• AutoLoss: Automated Loss Function Search in Recommendations
• Pre-trained Language Model for Web-scale Retrieval in Baidu Search
• FINT: Field-aware INTeraction Neural Network For CTR Prediction
• Random Offset Block Embedding Array (ROBE) for CriteoTB Benchmark MLPerf DLRM Model : 1000$\times$ Compression and 3.1$\times$ Faster Inference
• Learning Elastic Embeddings for Customizing On-Device Recommenders
• Model Size Reduction Using Frequency Based Double Hashing for Recommender Systems
• Learning Effective and Efficient Embedding via an Adaptively-Masked Twins-based Layer