45 citations · 62 across the 10 of their papers we have counts for
11 papers
Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs
Haoyu Zhang, Arshad Rahman, Adnan Aziz +55
Meta deploys a heterogeneous fleet comprising NVIDIA GPUs, AMD GPUs, and proprietary MTIA accelerators to reduce costs and mitigate vendor lock-in. However, the extensive history o…
Optimus: A Generic Operator-Level PyTorch Model Transformation Framework
Menglu Yu, Jiaqi Xu, Yuzhen Huang +19
In large-scale industrial applications, deep learning models that power recommendation and ranking have complex and diverse model architectures. These models are continuously devel…
GR2 Technical Report
Yufei Li, Zaiwei Zhang, Mingfu Liang +67
Industrial recommendation systems serve billions of users through a multi-stage funnel -- retrieval, early-stage ranking, and re-ranking -- where the final re-ranking step dispropo…
LoopFM: Learning frOm HistOrical RePresentations of Foundation Model for Recommendation
Shali Jiang, Hua Zheng, Boyang Liu +46
Knowledge distillation (KD) transfers a single scalar prediction from a large foundation model (FM) to compact vertical models (VMs), suffering from diminishing transfer ratio -- t…
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
DeepSeek-AI, Anyi Xu, Bangcai Lin +315
We present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models -- DeepSeek-V4-Pro with 1.6T parameters (49B activated) and DeepSe…
Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design
Bojian Hou, Xiaolong Liu, Xiaoyi Liu +26
Deriving predictable scaling laws that govern the relationship between model performance and computational investment is crucial for designing and allocating resources in massive-s…
Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models
Daochen Zha, Louis Feng, Liang Luo +8
Sharding a large machine learning model across multiple devices to balance the costs is important in distributed training. This is challenging because partitioning is NP-hard, and…
Training Recommender Systems at Scale: Communication-Efficient Model and Data Parallelism
Vipul Gupta, Dhruv Choudhary, Ping Tak Peter Tang +6
In this paper, we consider hybrid parallelism -- a paradigm that employs both Data Parallelism (DP) and Model Parallelism (MP) -- to scale distributed training of large recommendat…
TensorOpt: Exploring the Tradeoffs in Distributed DNN Training with Auto-Parallelism
Zhenkun Cai, Kaihao Ma, Xiao Yan +5
A good parallelization strategy can significantly improve the efficiency or reduce the cost for the distributed training of deep neural networks (DNNs). Recently, several methods h…
Efficient Processing of Reachability and Time-Based Path Queries in a Temporal Graph
Huanhuan Wu, Yuzhen Huang, James Cheng +2
A temporal graph is a graph in which vertices communicate with each other at specific time, e.g., calls at 11 a.m. and talks for 7 minutes, which is modeled by an edge from…
Efficient Processing of Very Large Graphs in a Small Cluster
Da Yan, Yuzhen Huang, James Cheng +1
Inspired by the success of Google's Pregel, many systems have been developed recently for iterative computation over big graphs. These systems provide a user-friendly vertex-centri…