12 citations · 29 across the 5 of their papers we have counts for
5 papers
FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
Zihao Ye, Lequn Chen, Ruihang Lai +8
Transformers, driven by attention mechanisms, form the foundation of large language models (LLMs). As these models scale up, efficient GPU attention kernels become essential for hi…
Relax: Composable Abstractions for End-to-End Dynamic Machine Learning
Ruihang Lai, Junru Shao, Siyuan Feng +16
Dynamic shape computations have become critical in modern machine learning workloads, especially in emerging large language models. The success of these models has driven the deman…
TensorIR: An Abstraction for Automatic Tensorized Program Optimization
Siyuan Feng, Bohan Hou, Hongyi Jin +8
Deploying deep learning models on various devices has become an important topic. The wave of hardware specialization brings a diverse set of acceleration primitives for multi-dimen…
Tensor Program Optimization with Probabilistic Programs
Junru Shao, Xiyou Zhou, Siyuan Feng +7
Automatic optimization for tensor programs becomes increasingly important as we deploy deep learning in various environments, and efficient optimization relies on a rich search spa…
Recurrent Residual Module for Fast Inference in Videos
Bowen Pan, Wuwei Lin, Xiaolin Fang +3
Deep convolutional neural networks (CNNs) have made impressive progress in many video recognition tasks such as video pose estimation and video object detection. However, CNN infer…