394 citations · 1.1k across the 21 of their papers we have counts for
21 papers
MTrainS: Improving DLRM training efficiency using heterogeneous memories
Hiwot Tadese Kassa, Paul Johnson, Jason Akers +7
Recommendation models are very large, requiring terabytes (TB) of memory during training. In pursuit of better quality, the model size and complexity grow over time, which requires…
With Shared Microexponents, A Little Shifting Goes a Long Way
Bita Rouhani, Ritchie Zhao, Venmugil Elango +19
This paper introduces Block Data Representations (BDR), a framework for exploring and evaluating a wide spectrum of narrow-precision formats for deep learning. It enables compariso…
DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction
Buyun Zhang, Liang Luo, Xi Liu +14
Learning feature interactions is important to the model performance of online advertising services. As a result, extensive efforts have been devoted to designing effective architec…
First-Generation Inference Accelerator Deployment at Facebook
Michael Anderson, Benny Chen, Stephen Chen +112
In this paper, we provide a deep dive into the deployment of inference accelerators at Facebook. Many of our ML workloads have unique characteristics, such as sparse memory accesse…
Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale
Zhaoxia, Deng, Jongsoo Park +17
Tremendous success of machine learning (ML) and the unabated growth in ML model complexity motivated many ML-specific designs in both CPU and accelerator architectures to speed up…
Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems
Xiaocong Du, Bhargav Bhushanam, Jiecao Yu +7
Deep learning recommendation systems at scale have provided remarkable gains through increasing model capacity (i.e. wider and deeper neural networks), but it comes at significant…
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Dheevatsa Mudigere, Yuchen Hao, Jianyu Huang +50
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand…
Efficient Soft-Error Detection for Low-precision Deep Learning Recommendation Models
Sihuan Li, Jianyu Huang, Ping Tak Peter Tang +4
Soft error, namely silent corruption of signal or datum in a computer system, cannot be caverlierly ignored as compute and communication density grow exponentially. Soft error dete…
FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference
Daya Khudia, Jianyu Huang, Protonu Basu +4
Deep learning models typically use single-precision (FP32) floating point data types for representing activations and weights, but a slew of recent research work has shown that com…
Mixed-Precision Embedding Using a Cache
Jie Amy Yang, Jianyu Huang, Jongsoo Park +2
In recommendation systems, practitioners observed that increase in the number of embedding tables and their sizes often leads to significant improvement in model performances. Give…
Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data
Mao Ye, Dhruv Choudhary, Jiecao Yu +6
Large scale deep learning provides a tremendous opportunity to improve the quality of content recommendation systems by employing both wider and deeper models, but this comes at gr…
Post-Training 4-bit Quantization on Embedding Tables
Hui Guan, Andrey Malevich, Jiyan Yang +2
Continuous representations have been widely adopted in recommender systems where a large number of entities are represented using embedding vectors. As the cardinality of the entit…
A Study of BFLOAT16 for Deep Learning Training
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi +16
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training acros…
Deep Learning Recommendation Model for Personalization and Recommendation Systems
Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi +21
With the advent of deep learning, neural network-based recommendation models have emerged as an important tool for tackling personalization and recommendation tasks. These networks…
Spatial-Winograd Pruning Enabling Sparse Winograd Convolution
Jiecao Yu, Jongsoo Park, Maxim Naumov
Deep convolutional neural networks (CNNs) are deployed in various applications but demand immense computational requirements. Pruning techniques and Winograd convolution are two ty…
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
Jongsoo Park, Maxim Naumov, Protonu Basu +25
The application of deep learning techniques resulted in remarkable improvement of machine learning models. In this paper provides detailed characterizations of deep learning models…
On Periodic Functions as Regularizers for Quantization of Neural Networks
Maxim Naumov, Utku Diril, Jongsoo Park +3
Deep learning models have been successfully used in computer vision and many other fields. We propose an unorthodox algorithm for performing quantization of the model parameters. I…
Glow: Graph Lowering Compiler Techniques for Neural Networks
Nadav Rotem, Jordan Fix, Saleem Abdulrasool +15
This paper presents the design of Glow, a machine learning compiler for heterogeneous hardware. It is a pragmatic approach to compilation that enables the generation of highly opti…
Two-step approach to scheduling quantum circuits
Gian Giacomo Guerreschi, Jongsoo Park
As the effort to scale up existing quantum hardware proceeds, it becomes necessary to schedule quantum gates in a way that minimizes the number of operations. There are three const…
Enabling Sparse Winograd Convolution by Native Pruning
Sheng Li, Jongsoo Park, Ping Tak Peter Tang
Sparse methods and the use of Winograd convolutions are two orthogonal approaches, each of which significantly accelerates convolution computations in modern CNNs. Sparse Winograd…
Faster CNNs with Direct Sparse Convolutions and Guided Pruning
Jongsoo Park, Sheng Li, Wei Wen +4
Phenomenally successful in practical inference problems, convolutional neural networks (CNN) are widely deployed in mobile devices, data centers, and even supercomputers. The numbe…