39 citations · 101 across the 14 of their papers we have counts for
11 papers · 1 filter
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
Ranggi Hwang, Jianyu Wei, Shijie Cao +4
Large language models (LLMs) based on transformers have made significant strides in recent years, the success of which is driven by scaling up their model size. Despite their high…
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Junyan Li, Li Lyna Zhang, Jiahang Xu +9
Deploying pre-trained transformer models like BERT on downstream tasks in resource-constrained scenarios is challenging due to their high inference cost, which grows rapidly with i…
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li +7
Automatic Speech Recognition (ASR) has seen remarkable advancements with deep neural networks, such as Transformer and Conformer. However, these models typically have large model s…
Integer or Floating Point? New Outlooks for Low-Bit Quantization on Large Language Models
Yijia Zhang, Lingran Zhao, Shijie Cao +6
Efficient deployment of large language models (LLMs) necessitates low-bit quantization to minimize model size and inference cost. While low-bit integer formats (e.g., INT8/INT4) ha…
ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
Chen Tang, Li Lyna Zhang, Huiqiang Jiang +6
Neural Architecture Search (NAS) has shown promising performance in the automatic design of vision transformers (ViT) exceeding 1G FLOPs. However, designing lightweight and low-lat…
Gamify Stencil Dwarf on Cloud for Democratizing Scientific Computing
Kun Li, Zhichun Li, Yuetao Chen +7
Stencil computation is one of the most important kernels in various scientific computing. Nowadays, most Stencil-driven scientific computing still relies heavily on supercomputers,…
SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
Li Lyna Zhang, Xudong Wang, Jiahang Xu +6
The combination of Neural Architecture Search (NAS) and quantization has proven successful in automatically designing low-FLOPs INT8 quantized neural networks (QNN). However, direc…
IRGen: Generative Modeling for Image Retrieval
Yidan Zhang, Ting Zhang, Dong Chen +11
While generative modeling has become prevalent across numerous research fields, its integration into the realm of image retrieval remains largely unexplored and underjustified. In…
LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup
Xiaohu Tang, Yang Wang, Ting Cao +5
On-device Deep Neural Network (DNN) inference consumes significant computing resources and development efforts. To alleviate that, we propose LUT-NN, the first system to empower in…
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
Zhiqi Lin, Youshan Miao, Guodong Liu +10
With the growing model size, deep neural networks (DNN) are increasingly trained over massive GPU accelerators, which demands a proper parallelization plan that transforms a DNN mo…
PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation
Ningxin Zheng, Huiqiang Jiang, Quanlu Zhang +8
Dynamic sparsity, where the sparsity patterns are unknown until runtime, poses a significant challenge to deep learning. The state-of-the-art sparsity-aware deep learning solutions…