110 citations · 152 across the 13 of their papers we have counts for
13 papers
CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution
Zihao Ye, Yingyi Huang, Hongyi Jin +11
GPU kernel agents and GPU programming languages have advanced separately, leaving expert kernels difficult to reproduce. Agents usually treat the compiler as a fixed black box and…
PithTrain: A Compact and Agent-Native MoE Training System
Ruihang Lai, Hao Kang, Haozhan Tang +6
Mixture-of-Experts (MoE) has become the dominant architecture for frontier language models. To meet this demand, production frameworks have built optimized MoE training stacks over…
Relax: Composable Abstractions for End-to-End Dynamic Machine Learning
Ruihang Lai, Junru Shao, Siyuan Feng +16
Dynamic shape computations have become critical in modern machine learning workloads, especially in emerging large language models. The success of these models has driven the deman…
RAF: Holistic Compilation for Deep Learning Model Training
Cody Hao Yu, Haozheng Fan, Guangtai Huang +9
As deep learning is pervasive in modern applications, many deep learning frameworks are presented for deep learning practitioners to develop and train DNN models rapidly. Meanwhile…
SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning
Zihao Ye, Ruihang Lai, Junru Shao +2
Sparse tensors are rapidly becoming critical components of modern deep learning workloads. However, developing high-performance sparse operators can be difficult and tedious, and e…
TensorIR: An Abstraction for Automatic Tensorized Program Optimization
Siyuan Feng, Bohan Hou, Hongyi Jin +8
Deploying deep learning models on various devices has become an important topic. The wave of hardware specialization brings a diverse set of acceleration primitives for multi-dimen…
Tensor Program Optimization with Probabilistic Programs
Junru Shao, Xiyou Zhou, Siyuan Feng +7
Automatic optimization for tensor programs becomes increasingly important as we deploy deep learning in various environments, and efficient optimization relies on a rich search spa…
Deep Neural Networks with Multi-Branch Architectures Are Less Non-Convex
Hongyang Zhang, Junru Shao, Ruslan Salakhutdinov
Several recently proposed architectures of neural networks such as ResNeXt, Inception, Xception, SqueezeNet and Wide ResNet are based on the designing idea of having multiple branc…
Learning to Ask: Neural Question Generation for Reading Comprehension
Xinya Du, Junru Shao, Claire Cardie
We study automatic question generation for sentences from text passages in reading comprehension. We introduce an attention-based sequence learning model for the task and investiga…
Generalized residual vector quantization for large scale data
Shicong Liu, Junru Shao, Hongtao Lu
Vector quantization is an essential tool for tasks involving large scale data, for example, large scale similarity search, which is crucial for content-based information retrieval…
Accelerated Distance Computation with Encoding Tree for High Dimensional Data
Shicong Liu, Junru Shao, Hongtao Lu
We propose a novel distance to calculate distance between high dimensional vector pairs, utilizing vector quantization generated encodings. Vector quantization based methods are su…
Improved Residual Vector Quantization for High-dimensional Approximate Nearest Neighbor Search
Shicong Liu, Hongtao Lu, Junru Shao
Quantization methods have been introduced to perform large scale approximate nearest search tasks. Residual Vector Quantization (RVQ) is one of the effective quantization methods.…
HCLAE: High Capacity Locally Aggregating Encodings for Approximate Nearest Neighbor Search
Shicong Liu, Junru Shao, Hongtao Lu
Vector quantization-based approaches are successful to solve Approximate Nearest Neighbor (ANN) problems which are critical to many applications. The idea is to generate effective…