4 papers
Coordination-Free Workload Regularization for Variable-Cost Distributed Training
Tao Huang, Chen Zhu, Antonio Alonso +4
Synchronous data-parallel training usually assigns every rank the same number of records, even when record costs vary by orders of magnitude. For variable-length models, this conve…
Scalable Hyperparameter-Divergent Ensemble Training with Automatic Learning Rate Exploration for Large Models
Hailing Cheng, Tao Huang, Chen Zhu +1
Training large neural networks with data-parallel stochastic gradient descent allocates N GPU replicas to compute effectively identical updates -- a practice that leaves the rich s…
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
Lars Hertel, Gaurav Srivastava, Syed Ali Naqvi +21
LinkedIn Feed enables professionals worldwide to discover relevant content, build connections, and share knowledge at scale. We present Feed Sequential Recommender (Feed SR), a tra…
Large Scale Retrieval for the LinkedIn Feed using Causal Language Models
Sudarshan Srinivasa Ramanujam, Antonio Alonso, Saurabh Kataria +20
In large scale recommendation systems like the LinkedIn Feed, the retrieval stage is critical for narrowing hundreds of millions of potential candidates to a manageable subset for…