51 citations · 273 across the 38 of their papers we have counts for
7 papers · 1 filter
Foundation Model is Efficient Multimodal Multitask Model Selector
Fanqing Meng, Wenqi Shao, Zhanglin Peng +4
This paper investigates an under-explored but important problem: given a collection of pre-trained neural networks, predicting their performance on each multi-modal task without fi…
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang +7
Large language models (LLMs) have revolutionized natural language processing tasks. However, their practical deployment is hindered by their immense memory and computation requirem…
Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space
Wenqi Shao, Xun Zhao, Yixiao Ge +5
This paper addresses an important problem of ranking the pre-trained deep neural networks and screening the most transferable ones for downstream tasks. It is challenging because t…
Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution
Zhang Zhaoyang, Shao Wenqi, Gu Jinwei +2
Model quantization is challenging due to many tedious hyper-parameters such as precision (bitwidth), dynamic range (minimum and maximum discrete values) and stepsize (interval betw…
BWCP: Probabilistic Learning-to-Prune Channels for ConvNets via Batch Whitening
Wenqi Shao, Hang Yu, Zhaoyang Zhang +3
This work presents a probabilistic channel pruning method to accelerate Convolutional Neural Networks (CNNs). Previous pruning methods often zero out unimportant channels in traini…
Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning
Zhongzhan Huang, Wenqi Shao, Xinjiang Wang +2
Channel pruning is a popular technique for compressing convolutional neural networks (CNNs), where various pruning criteria have been proposed to remove the redundant filters. From…
Towards Understanding Regularization in Batch Normalization
Ping Luo, Xinjiang Wang, Wenqi Shao +1
Batch Normalization (BN) improves both convergence and generalization in training neural networks. This work understands these phenomena theoretically. We analyze BN by using a bas…