54 citations · 94 across the 9 of their papers we have counts for
9 papers
A System for Microserving of LLMs
Hongyi Jin, Ruihang Lai, Charlie F. Ruan +5
The recent advances in LLMs bring a strong demand for efficient system support to improve overall serving efficiency. As LLM inference scales towards multiple GPUs and even multipl…
Relax: Composable Abstractions for End-to-End Dynamic Machine Learning
Ruihang Lai, Junru Shao, Siyuan Feng +16
Dynamic shape computations have become critical in modern machine learning workloads, especially in emerging large language models. The success of these models has driven the deman…
ACRoBat: Optimizing Auto-batching of Dynamic Deep Learning at Compile Time
Pratik Fegade, Tianqi Chen, Phillip B. Gibbons +1
Dynamic control flow is an important technique often used to design expressive and efficient deep learning computations for applications such as text parsing, machine translation,…
ED-Batch: Efficient Automatic Batching of Dynamic Neural Networks via Learned Finite State Machines
Siyuan Chen, Pratik Fegade, Tianqi Chen +2
Batching has a fundamental influence on the efficiency of deep neural network (DNN) execution. However, for dynamic DNNs, efficient batching is particularly challenging as the data…
The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding
Pratik Fegade, Tianqi Chen, Phillip B. Gibbons +1
There is often variation in the shape and size of input data used for deep learning. In many cases, such data can be represented using tensors with non-uniform shapes, or ragged te…
Cortex: A Compiler for Recursive Deep Learning Models
Pratik Fegade, Tianqi Chen, Phillip B. Gibbons +1
Optimizing deep learning models is generally performed in two steps: (i) high-level graph optimizations such as kernel fusion and (ii) low level kernel optimizations such as those…
RowClone: Accelerating Data Movement and Initialization Using DRAM
Vivek Seshadri, Yoongu Kim, Chris Fallin +8
In existing systems, to perform any bulk data movement operation (copy or initialization), the data has to first be read into the on-chip processor, all the way into the L1 cache,…
Buddy-RAM: Improving the Performance and Efficiency of Bulk Bitwise Operations Using DRAM
Vivek Seshadri, Donghyuk Lee, Thomas Mullins +7
Bitwise operations are an important component of modern day programming. Many widely-used data structures (e.g., bitmap indices in databases) rely on fast bitwise operations on lar…
A Framework for Accelerating Bottlenecks in GPU Execution with Assist Warps
Nandita Vijaykumar, Gennady Pekhimenko, Adwait Jog +7
Modern Graphics Processing Units (GPUs) are well provisioned to support the concurrent execution of thousands of threads. Unfortunately, different bottlenecks during execution and…