5 papers
PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
Yuchen Yang, Yifan Zhao, Anisha Dasgupta +1
Mixture-of-Experts (MoE) is a popular class of large language models (LLMs), offering high efficiency and accuracy. However, in KV-cache-intensive serving scenarios, MoEs often exh…
ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs
Yuchen Yang, Yifan Zhao, Shubham Ugare +2
Mixed precision quantization has become an important technique for optimizing the execution of deep neural networks (DNNs). Certified robustness, which provides provable guarantees…
Neptune: Advanced ML Operator Fusion for Locality and Parallelism on GPUs
Yifan Zhao, Egan Johnson, Prasanth Chatarasi +2
Operator fusion has become a key optimization for deep learning, which combines multiple deep learning operators to improve data reuse and reduce global memory transfers. However,…
Nautilus: An Auto-Scheduling Tensor Compiler for Efficient Tiled GPU Kernels
Yifan Zhao, Yuchen Yang, Matei Budiu +1
We present Nautilus, a novel tensor compiler that moves toward fully automated math-to-kernel optimization. Nautilus compiles a high-level algebraic specification of tensor operato…
RiO-DETR: DETR for Real-time Oriented Object Detection
Zhangchi Hu, Yifan Zhao, Yansong Peng +8
We present RiO-DETR: DETR for Real-time Oriented Object Detection, the first real-time oriented detection transformer to the best of our knowledge. Adapting DETR to oriented boundi…