171 citations · 226 across the 9 of their papers we have counts for
3 papers · 1 filter
Packrat: Automatic Reconfiguration for Latency Minimization in CPU-based DNN Serving
Ankit Bhardwaj, Amar Phanishayee, Deepak Narayanan +2
In this paper, we investigate how to push the performance limits of serving Deep Neural Network (DNN) models on CPU-based servers. Specifically, we observe that while intra-operato…
MGit: A Model Versioning and Management System
Wei Hao, Daniel Mendoza, Rafael da Silva +2
Models derived from other models are extremely common in machine learning (ML) today. For example, transfer learning is used to create task-specific models from "pre-trained" model…
Cheaply Evaluating Inference Efficiency Metrics for Autoregressive Transformer APIs
Deepak Narayanan, Keshav Santhanam, Peter Henderson +3
Large language models (LLMs) power many state-of-the-art systems in natural language processing. However, these models are extremely computationally expensive, even at inference ti…