160 citations · 425 across the 18 of their papers we have counts for
18 papers
Efficient Stitchable Task Adaptation
Haoyu He, Zizheng Pan, Jing Liu +2
The paradigm of pre-training and fine-tuning has laid the foundation for deploying deep learning models. However, most fine-tuning methods are designed to meet a specific resource…
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
Jing Liu, Ruihao Gong, Xiuying Wei +3
Large Language Models (LLMs) excel in NLP, but their demands hinder their widespread deployment. While Quantization-Aware Training (QAT) offers a solution, its extensive training c…
Stitched ViTs are Flexible Vision Backbones
Zizheng Pan, Jing Liu, Haoyu He +2
Large pretrained plain vision Transformers (ViTs) have been the workhorse for many downstream tasks. However, existing works utilizing off-the-shelf ViTs are inefficient in terms o…
PTQD: Accurate Post-Training Quantization for Diffusion Models
Yefei He, Luping Liu, Jing Liu +3
Diffusion models have recently dominated image synthesis tasks. However, the iterative denoising process is expensive in computations at inference time, making diffusion models les…
A Survey on Efficient Training of Transformers
Bohan Zhuang, Jing Liu, Zizheng Pan +3
Recent advances in Transformers have come with a huge requirement on computing resources, highlighting the importance of developing efficient training techniques to make Transforme…
EcoFormer: Energy-Saving Attention with Linear Complexity
Jing Liu, Zizheng Pan, Haoyu He +2
Transformer is a transformative framework that models sequential data and has achieved remarkable performance on a wide range of tasks, but with high computational and energy cost.…
FocusFormer: Focusing on What We Need via Architecture Sampler
Jing Liu, Jianfei Cai, Bohan Zhuang
Vision Transformers (ViTs) have underpinned the recent breakthroughs in computer vision. However, designing the architectures of ViTs is laborious and heavily relies on expert know…
Dynamic Focus-aware Positional Queries for Semantic Segmentation
Haoyu He, Jianfei Cai, Zizheng Pan +4
The DETR-like segmentors have underpinned the most recent breakthroughs in semantic segmentation, which end-to-end train a set of queries representing the class prototypes or targe…
Pruning Self-attentions into Convolutional Layers in Single Path
Haoyu He, Jianfei Cai, Jing Liu +4
Vision Transformers (ViTs) have achieved impressive performance over various computer vision tasks. However, modeling global correlations with multi-head self-attention (MSA) layer…
Mesa: A Memory-saving Training Framework for Transformers
Zizheng Pan, Peng Chen, Haoyu He +3
There has been an explosion of interest in designing high-performance Transformers. While Transformers have delivered significant performance improvements, training such networks i…
Less is More: Pay Less Attention in Vision Transformers
Zizheng Pan, Bohan Zhuang, Haoyu He +2
Transformers have become one of the dominant architectures in deep learning, particularly as a powerful alternative to convolutional neural networks (CNNs) in computer vision. Howe…
Scalable Vision Transformers with Hierarchical Pooling
Zizheng Pan, Bohan Zhuang, Jing Liu +2
The recently proposed Visual image Transformers (ViT) with pure attention have achieved promising performance on image recognition tasks, such as image classification. However, the…
Single-path Bit Sharing for Automatic Loss-aware Model Compression
Jing Liu, Bohan Zhuang, Peng Chen +3
Network pruning and quantization are proven to be effective ways for deep model compression. To obtain a highly compact model, most methods first perform network pruning and then c…
AQD: Towards Accurate Fully-Quantized Object Detection
Peng Chen, Jing Liu, Bohan Zhuang +2
Network quantization allows inference to be conducted using low-precision arithmetic for improved inference efficiency of deep neural networks on edge devices. However, designing a…
Generative Low-bitwidth Data Free Quantization
Shoukai Xu, Haokun Li, Bohan Zhuang +4
Neural network quantization is an effective way to compress deep models and improve their execution latency and energy efficiency, so that they can be deployed on mobile or embedde…
Discrimination-aware Network Pruning for Deep Model Compression
Jing Liu, Bohan Zhuang, Zhuangwei Zhuang +4
We study network pruning which aims to remove redundant channels/kernels and hence speed up the inference of deep networks. Existing pruning methods either train from scratch with…
Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations
Bohan Zhuang, Jing Liu, Mingkui Tan +3
This paper tackles the problem of training a deep convolutional neural network of both low-bitwidth weights and activations. Optimizing a low-precision network is very challenging…
Discrimination-aware Channel Pruning for Deep Neural Networks
Zhuangwei Zhuang, Mingkui Tan, Bohan Zhuang +5
Channel pruning is one of the predominant approaches for deep model compression. Existing pruning methods either train from scratch with sparsity constraints on channels, or minimi…