571 citations · 1.8k across the 19 of their papers we have counts for
20 papers
Supporting Massive DLRM Inference Through Software Defined Memory
Ehsan K. Ardestani, Changkyu Kim, Seung Jae Lee +17
Deep Learning Recommendation Models (DLRM) are widespread, account for a considerable data center footprint, and grow by more than 1.5x per year. With model size soon to be in tera…
Differentiable NAS Framework and Application to Ads CTR Prediction
Ravi Krishna, Aravind Kalaiah, Bichen Wu +4
Neural architecture search (NAS) methods aim to automatically find the optimal deep neural network (DNN) architecture as measured by a given objective function, typically some comb…
Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale
Zhaoxia, Deng, Jongsoo Park +17
Tremendous success of machine learning (ML) and the unabated growth in ML model complexity motivated many ML-specific designs in both CPU and accelerator architectures to speed up…
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Dheevatsa Mudigere, Yuchen Hao, Jianyu Huang +50
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand…
FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference
Daya Khudia, Jianyu Huang, Protonu Basu +4
Deep learning models typically use single-precision (FP32) floating point data types for representing activations and weights, but a slew of recent research work has shown that com…
Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems
Maxim Naumov, John Kim, Dheevatsa Mudigere +12
Large-scale training is important to ensure high performance and accuracy of machine-learning models. At Facebook we use many different models, including computer vision, video and…
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
Liu Ke, Udit Gupta, Carole-Jean Wu +18
Personalized recommendation systems leverage deep learning models and account for the majority of data center AI cycles. Their performance is dominated by memory-bound sparse embed…
A Study of BFLOAT16 for Deep Learning Training
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi +16
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training acros…
The Architectural Implications of Facebook's DNN-based Personalized Recommendation
Udit Gupta, Carole-Jean Wu, Xiaodong Wang +12
The widespread application of deep learning has changed the landscape of computation in the data center. In particular, personalized recommendation for content ranking is now large…
Deep Learning Recommendation Model for Personalization and Recommendation Systems
Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi +21
With the advent of deep learning, neural network-based recommendation models have emerged as an important tool for tackling personalization and recommendation tasks. These networks…
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
Jongsoo Park, Maxim Naumov, Protonu Basu +25
The application of deep learning techniques resulted in remarkable improvement of machine learning models. In this paper provides detailed characterizations of deep learning models…
Bandana: Using Non-volatile Memory for Storing Deep Learning Models
Assaf Eisenman, Maxim Naumov, Darryl Gardner +5
Typical large-scale recommender systems use deep learning models that are stored on a large amount of DRAM. These models often rely on embeddings, which consume most of the require…
Glow: Graph Lowering Compiler Techniques for Neural Networks
Nadav Rotem, Jordan Fix, Saleem Abdulrasool +15
This paper presents the design of Glow, a machine learning compiler for heterogeneous hardware. It is a pragmatic approach to compilation that enables the generation of highly opti…
Practical optimization for hybrid quantum-classical algorithms
Gian Giacomo Guerreschi, Mikhail Smelyanskiy
A novel class of hybrid quantum-classical algorithms based on the variational approach have recently emerged from separate proposals addressing, for example, quantum chemistry and…
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal +2
The stochastic gradient descent (SGD) method and its variants are algorithms of choice for many Deep Learning tasks. These methods operate in a small-batch regime wherein a fractio…
Distributed Hessian-Free Optimization for Deep Neural Network
Xi He, Dheevatsa Mudigere, Mikhail Smelyanskiy +1
Training deep neural network is a high dimensional and a highly non-convex optimization problem. Stochastic gradient descent (SGD) algorithm and it's variations are the current sta…
High Performance Emulation of Quantum Circuits
Thomas Häner, Damian S. Steiger, Mikhail Smelyanskiy +1
As quantum computers of non-trivial size become available in the near future, it is imperative to develop tools to emulate small quantum computers. This allows for validation and d…
Error Sensitivity to Environmental Noise in Quantum Circuits for Chemical State Preparation
Nicolas P. D. Sawaya, Mikhail Smelyanskiy, Jarrod R. McClean +1
Calculating molecular energies is likely to be one of the first useful applications to achieve quantum supremacy, performing faster on a quantum than a classical computer. However,…
qHiPSTER: The Quantum High Performance Software Testing Environment
Mikhail Smelyanskiy, Nicolas P. D. Sawaya, Alán Aspuru-Guzik
We present qHiPSTER, the Quantum High Performance Software Testing Environment. qHiPSTER is a distributed high-performance implementation of a quantum simulator on a classical comp…
Lattice QCD with Domain Decomposition on Intel Xeon Phi Co-Processors
Simon Heybrock, Bálint Joó, Dhiraj D. Kalamkar +4
The gap between the cost of moving data and the cost of computing continues to grow, making it ever harder to design iterative solvers on extreme-scale architectures. This problem…