91 citations · 505 across the 35 of their papers we have counts for
13 papers · 1 filter
LoopFM: Learning frOm HistOrical RePresentations of Foundation Model for Recommendation
Shali Jiang, Hua Zheng, Boyang Liu +46
Knowledge distillation (KD) transfers a single scalar prediction from a large foundation model (FM) to compact vertical models (VMs), suffering from diminishing transfer ratio -- t…
Multi-Objective Bilevel Learning
Zhiyao Zhang, Zhuqing Liu, Xin Zhang +3
As machine learning (ML) applications grow increasingly complex in recent years, modern ML frameworks often need to address multiple potentially conflicting objectives with coupled…
Hierarchical LoRA MoE for Efficient CTR Model Scaling
Zhichen Zeng, Mengyue Hang, Xiaolong Liu +11
Deep models have driven significant advances in click-through rate (CTR) prediction. While vertical scaling via layer stacking improves model expressiveness, the layer-by-layer seq…
CubicML: Automated ML for Large ML Systems Co-design with ML Prediction of Performance
Wei Wen, Quanyu Zhu, Weiwei Chu +2
Scaling up deep learning models has been proven effective to improve intelligence of machine learning (ML) models, especially for industry recommendation models and large language…
Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale
Wei Wen, Kuang-Hung Liu, Igor Fedorov +19
Neural Architecture Search (NAS) has demonstrated its efficacy in computer vision and potential for ranking systems. However, prior work focused on academic problems, which are eva…
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery
Kiwan Maeng, Shivam Bharuka, Isabel Gao +8
The paper proposes and optimizes a partial recovery training system, CPR, for recommendation models. CPR relaxes the consistency requirement by enabling non-failed nodes to proceed…
Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data
Mao Ye, Dhruv Choudhary, Jiecao Yu +6
Large scale deep learning provides a tremendous opportunity to improve the quality of content recommendation systems by employing both wider and deeper models, but this comes at gr…
ShadowSync: Performing Synchronization in the Background for Highly Scalable Distributed Training
Qinqing Zheng, Bor-Yiing Su, Jiyan Yang +7
Recommendation systems are often trained with a tremendous amount of data, and distributed training is the workhorse to shorten the training time. While the training throughput can…
Post-Training 4-bit Quantization on Embedding Tables
Hui Guan, Andrey Malevich, Jiyan Yang +2
Continuous representations have been widely adopted in recommender systems where a large number of entities are represented using embedding vectors. As the cardinality of the entit…
Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems
Antonio Ginart, Maxim Naumov, Dheevatsa Mudigere +2
Embedding representations power machine intelligence in many applications, including recommendation systems, but they are space intensive -- potentially occupying hundreds of gigab…
Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
Hao-Jun Michael Shi, Dheevatsa Mudigere, Maxim Naumov +1
Modern deep learning-based recommendation systems exploit hundreds to thousands of different categorical features, each with millions of different categories ranging from clicks to…
A Study of BFLOAT16 for Deep Learning Training
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi +16
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training acros…
Tensor machines for learning target-specific polynomial features
Jiyan Yang, Alex Gittens
Recent years have demonstrated that using random feature maps can significantly decrease the training and testing times of kernel-based algorithms without significantly lowering th…