81 citations · 144 across the 7 of their papers we have counts for
7 papers
Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving
Yi Xu, Ehsan K. Ardestani, Wenyin Fu +7
As serving capacity demand surpasses that of training, serving efficiency becomes increasingly important. Prefill-decode (P/D) disaggregation improves serving efficiency through sp…
SMaLL: A Software Framework for portable Machine Learning Libraries
Upasana Sridhar, Nicholai Tukanov, Elliott Binder +3
Interest in deploying Deep Neural Network (DNN) inference on edge devices has resulted in an explosion of the number and types of hardware platforms to use. While the high-level pr…
First-Generation Inference Accelerator Deployment at Facebook
Michael Anderson, Benny Chen, Stephen Chen +112
In this paper, we provide a deep dive into the deployment of inference accelerators at Facebook. Many of our ML workloads have unique characteristics, such as sparse memory accesse…
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
Liu Ke, Udit Gupta, Carole-Jean Wu +18
Personalized recommendation systems leverage deep learning models and account for the majority of data center AI cycles. Their performance is dominated by memory-bound sparse embed…
A Flexible Framework for Parallel Multi-Dimensional DFTs
Doru Thom Popovici, Martin D. Schatz, Franz Franchetti +1
Multi-dimensional discrete Fourier transforms (DFT) are typically decomposed into multiple 1D transforms. Hence, parallel implementations of any multi-dimensional DFT focus on para…
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
Jongsoo Park, Maxim Naumov, Protonu Basu +25
The application of deep learning techniques resulted in remarkable improvement of machine learning models. In this paper provides detailed characterizations of deep learning models…
Exploiting Symmetry in Tensors for High Performance: Multiplication with Symmetric Tensors
Martin D. Schatz, Tze Meng Low, Robert A. van de Geijn +1
Symmetric tensor operations arise in a wide variety of computations. However, the benefits of exploiting symmetry in order to reduce storage and computation is in conflict with a d…