most citedLite Transformer with Long-Short Range Attention

130 citations · 247 across the 6 of their papers we have counts for

collaborators

6 papers

cs.CV202036 cited

Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution

Haotian Tang, Zhijian Liu, Shengyu Zhao +4

Self-driving cars need to understand 3D scenes efficiently and accurately in order to drive safely. Given the limited hardware resources, existing 3D perception models are not able…

cs.CV202016 cited

Hardware-Centric AutoML for Mixed-Precision Quantization

Kuan Wang, Zhijian Liu, Yujun Lin +2

Model quantization is a widely used technique to compress and accelerate deep neural network (DNN) inference. Emergent DNN hardware accelerators begin to support mixed precision (1…

cs.LG202022 cited

APQ: Joint Search for Network Architecture, Pruning and Quantization Policy

Tianzhe Wang, Kuan Wang, Han Cai +3

We present APQ for efficient deep learning inference on resource-constrained hardware. Unlike previous methods that separately search the neural architecture, pruning policy, and q…

cs.CL2020130 cited

Lite Transformer with Long-Short Range Attention

Zhanghao Wu, Zhijian Liu, Ji Lin +2

Transformer has become ubiquitous in natural language processing (e.g., machine translation, question answering); however, it requires enormous amount of computations to achieve hi…

cs.CV20199 cited

Training Kinetics in 15 Minutes: Large-scale Distributed Training on Videos

Ji Lin, Chuang Gan, Song Han

Deep video recognition is more computationally expensive than image recognition, especially on large-scale datasets like Kinetics [1]. Therefore, training scalability is essential…

cs.LG201934 cited

Defensive Quantization: When Efficiency Meets Robustness

Ji Lin, Chuang Gan, Song Han

Neural network quantization is becoming an industry standard to efficiently deploy deep learning models on hardware platforms, such as CPU, GPU, TPU, and FPGAs. However, we observe…