394 citations · 1.4k across the 41 of their papers we have counts for
44 papers
Design Space Exploration and Optimization for Carbon-Efficient Extended Reality Systems
Mariam Elgamal, Doug Carmean, Elnaz Ansari +8
As computing hardware becomes more specialized, designing environmentally sustainable computing systems requires accounting for both hardware and software parameters. Our goal is t…
READ: Recurrent Adaptation of Large Transformers
John Nguyen, Sid Wang, Ke Li +1
Fine-tuning large-scale Transformers has led to the explosion of many AI applications across Natural Language Processing and Computer Vision tasks. However, fine-tuning all pre-tra…
GreenScale: Carbon-Aware Systems for Edge Computing
Young Geun Kim, Udit Gupta, Andrew McCrabb +4
To improve the environmental implications of the growing demand of computing, future applications need to improve the carbon-efficiency of computing infrastructures. State-of-the-a…
Green Federated Learning
Ashkan Yousefpour, Shen Guo, Ashish Shenoy +7
The rapid progress of AI is fueled by increasingly large and computationally intensive machine learning models and datasets. As a consequence, the amount of compute used in trainin…
Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference
Haiyang Huang, Newsha Ardalani, Anna Sun +6
Mixture-of-Experts (MoE) models have gained popularity in achieving state-of-the-art performance in a wide range of tasks in computer vision and natural language processing. They e…
MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendation
Samuel Hsia, Udit Gupta, Bilge Acun +5
Deep learning recommendation systems serve personalized content under diverse tail-latency targets and input-query loads. In order to do so, state-of-the-art recommendation models…
FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Models
Geet Sethi, Pallab Bhattacharya, Dhruv Choudhary +2
Sequence-based deep learning recommendation models (DLRMs) are an emerging class of DLRMs showing great improvements over their prior sum-pooling based counterparts at capturing us…
FedGPO: Heterogeneity-Aware Global Parameter Optimization for Efficient Federated Learning
Young Geun Kim, Carole-Jean Wu
Federated learning (FL) has emerged as a solution to deal with the risk of privacy leaks in machine learning training. This approach allows a variety of mobile devices to collabora…
RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure
Mark Zhao, Dhruv Choudhary, Devashish Tyagi +9
We present RecD (Recommendation Deduplication), a suite of end-to-end infrastructure optimizations across the Deep Learning Recommendation Model (DLRM) training pipeline. RecD addr…
Understanding the Power of Evolutionary Computation for GPU Code Optimization
Jhe-Yu Liou, Muaaz Awan, Steven Hofmeyr +2
Achieving high performance for GPU codes requires developers to have significant knowledge in parallel programming and GPU architectures, and in-depth understanding of the applicat…
Understanding Scaling Laws for Recommendation Models
Newsha Ardalani, Carole-Jean Wu, Zeliang Chen +2
Scale has been a major driving force in improving machine learning performance, and understanding scaling laws is essential for strategic planning for a sustainable model quality p…
DataPerf: Benchmarks for Data-Centric AI Development
Mark Mazumder, Colby Banbury, Xiaozhe Yao +42
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithf…
FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learning
Meisam Hejazinia, Dzmitry Huba, Ilias Leontiadis +7
Federated learning (FL) has emerged as an effective approach to address consumer privacy needs. FL has been successfully applied to certain machine learning tasks, such as training…
Infinite Recommendation Networks: A Data-Centric Approach
Noveen Sachdeva, Mehak Preet Dhaliwal, Carole-Jean Wu +1
We leverage the Neural Tangent Kernel and its equivalence to training infinitely-wide neural networks to devise -AE: an autoencoder with infinitely-wide bottleneck layers.…
Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity
Kiwan Maeng, Haiyu Lu, Luca Melis +3
Federated learning (FL) is an effective mechanism for data privacy in recommender systems by running machine learning model training on-device. While prior FL optimizations tackled…
Hercules: Heterogeneity-Aware Inference Serving for At-Scale Personalized Recommendation
Liu Ke, Udit Gupta, Mark Hempstead +3
Personalized recommendation is an important class of deep-learning applications that powers a large collection of internet services and consumes a considerable amount of datacenter…
RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation
Geet Sethi, Bilge Acun, Niket Agarwal +3
We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two ke…
On Sampling Collaborative Filtering Datasets
Noveen Sachdeva, Carole-Jean Wu, Julian McAuley
We study the practical consequences of dataset sampling strategies on the ranking performance of recommendation algorithms. Recommender systems are generally trained and evaluated…
Carbon Explorer: A Holistic Approach for Designing Carbon Aware Datacenters
Bilge Acun, Benjamin Lee, Fiodar Kazhamiaka +5
Technology companies have been leading the way to a renewable energy transformation, by investing in renewable energy sources to reduce the carbon footprint of their datacenters. I…
Papaya: Practical, Private, and Scalable Federated Learning
Dzmitry Huba, John Nguyen, Kshitiz Malik +11
Cross-device Federated Learning (FL) is a distributed learning paradigm with several challenges that differentiate it from traditional distributed learning, variability in the syst…
Sustainable AI: Environmental Implications, Challenges and Opportunities
Carole-Jean Wu, Ramya Raghavendra, Udit Gupta +22
This paper explores the environmental impact of the super-linear growth trends for AI from a holistic perspective, spanning Data, Algorithms, and System Hardware. We characterize t…
Socio-Technological Challenges and Opportunities: Paths Forward
Carole-Jean Wu, Srilatha Manne, Parthasarathy Ranganathan +2
Advancements in digital technologies have a bootstrapping effect. The past fifty years of technological innovations from the computer architecture community have brought innovation…
Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training
Mark Zhao, Niket Agarwal, Aarti Basant +14
Datacenter-scale AI training clusters consisting of thousands of domain-specific accelerators (DSA) are used to train increasingly-complex deep learning models. These clusters rely…
AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning
Young Geun Kim, Carole-Jean Wu
Federated learning enables a cluster of decentralized mobile devices at the edge to collaboratively train a shared machine learning model, while keeping all the raw training sample…