394 citations · 1.4k across the 41 of their papers we have counts for
18 papers · 1 filter
READ: Recurrent Adaptation of Large Transformers
John Nguyen, Sid Wang, Ke Li +1
Fine-tuning large-scale Transformers has led to the explosion of many AI applications across Natural Language Processing and Computer Vision tasks. However, fine-tuning all pre-tra…
Green Federated Learning
Ashkan Yousefpour, Shen Guo, Ashish Shenoy +7
The rapid progress of AI is fueled by increasingly large and computationally intensive machine learning models and datasets. As a consequence, the amount of compute used in trainin…
FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Models
Geet Sethi, Pallab Bhattacharya, Dhruv Choudhary +2
Sequence-based deep learning recommendation models (DLRMs) are an emerging class of DLRMs showing great improvements over their prior sum-pooling based counterparts at capturing us…
FedGPO: Heterogeneity-Aware Global Parameter Optimization for Efficient Federated Learning
Young Geun Kim, Carole-Jean Wu
Federated learning (FL) has emerged as a solution to deal with the risk of privacy leaks in machine learning training. This approach allows a variety of mobile devices to collabora…
RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure
Mark Zhao, Dhruv Choudhary, Devashish Tyagi +9
We present RecD (Recommendation Deduplication), a suite of end-to-end infrastructure optimizations across the Deep Learning Recommendation Model (DLRM) training pipeline. RecD addr…
DataPerf: Benchmarks for Data-Centric AI Development
Mark Mazumder, Colby Banbury, Xiaozhe Yao +42
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithf…
RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation
Geet Sethi, Bilge Acun, Niket Agarwal +3
We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two ke…
On Sampling Collaborative Filtering Datasets
Noveen Sachdeva, Carole-Jean Wu, Julian McAuley
We study the practical consequences of dataset sampling strategies on the ranking performance of recommendation algorithms. Recommender systems are generally trained and evaluated…
Papaya: Practical, Private, and Scalable Federated Learning
Dzmitry Huba, John Nguyen, Kshitiz Malik +11
Cross-device Federated Learning (FL) is a distributed learning paradigm with several challenges that differentiate it from traditional distributed learning, variability in the syst…
Sustainable AI: Environmental Implications, Challenges and Opportunities
Carole-Jean Wu, Ramya Raghavendra, Udit Gupta +22
This paper explores the environmental impact of the super-linear growth trends for AI from a holistic perspective, spanning Data, Algorithms, and System Hardware. We characterize t…
AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning
Young Geun Kim, Carole-Jean Wu
Federated learning enables a cluster of decentralized mobile devices at the edge to collaboratively train a shared machine learning model, while keeping all the raw training sample…
Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale
Zhaoxia, Deng, Jongsoo Park +17
Tremendous success of machine learning (ML) and the unabated growth in ML model complexity motivated many ML-specific designs in both CPU and accelerator architectures to speed up…
TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models
Chunxing Yin, Bilge Acun, Xing Liu +1
The memory capacity of embedding tables in deep learning recommendation models (DLRMs) is increasing dramatically from tens of GBs to TBs across the industry. Given the fast growth…
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery
Kiwan Maeng, Shivam Bharuka, Isabel Gao +8
The paper proposes and optimizes a partial recovery training system, CPR, for recommendation models. CPR relaxes the consistency requirement by enabling non-failed nodes to proceed…
AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance
Young Geun Kim, Carole-Jean Wu
Deep learning inference is increasingly run at the edge. As the programming and system stack support becomes mature, it enables acceleration opportunities within a mobile system, w…
Developing a Recommendation Benchmark for MLPerf Training and Inference
Carole-Jean Wu, Robin Burke, Ed H. Chi +4
Deep learning-based recommendation models are used pervasively and broadly, for example, to recommend movies, products, or other information most relevant to users, in order to enh…
MLPerf Training Benchmark
Peter Mattson, Christine Cheng, Cody Coleman +34
Machine learning (ML) needs industry-standard performance benchmarks to support design and competitive evaluation of the many emerging software and hardware solutions for ML. But M…
Exploiting Parallelism Opportunities with Deep Learning Frameworks
Yu Emma Wang, Carole-Jean Wu, Xiaodong Wang +2
State-of-the-art machine learning frameworks support a wide variety of design features to enable a flexible machine learning programming interface and to ease the programmability b…