24 citations · 77 across the 12 of their papers we have counts for
12 papers
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang +7
Large language models (LLMs) have revolutionized natural language processing tasks. However, their practical deployment is hindered by their immense memory and computation requirem…
Real-time Controllable Denoising for Image and Video
Zhaoyang Zhang, Yitong Jiang, Wenqi Shao +4
Controllable image denoising aims to generate clean samples with human perceptual priors and balance sharpness and smoothness. In traditional filter-based denoising methods, this c…
Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space
Wenqi Shao, Xun Zhao, Yixiao Ge +5
This paper addresses an important problem of ranking the pre-trained deep neural networks and screening the most transferable ones for downstream tasks. It is challenging because t…
Semantic-preserved Communication System for Highly Efficient Speech Transmission
Tianxiao Han, Qianqian Yang, Zhiguo Shi +2
Deep learning (DL) based semantic communication methods have been explored for the efficient transmission of images, text, and speech in recent years. In contrast to traditional wi…
Dynamic Token Normalization Improves Vision Transformers
Wenqi Shao, Yixiao Ge, Zhaoyang Zhang +4
Vision Transformer (ViT) and its variants (e.g., Swin, PVT) have achieved great success in various computer vision tasks, owing to their capability to learn long-range contextual i…
Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution
Zhang Zhaoyang, Shao Wenqi, Gu Jinwei +2
Model quantization is challenging due to many tedious hyper-parameters such as precision (bitwidth), dynamic range (minimum and maximum discrete values) and stepsize (interval betw…
BWCP: Probabilistic Learning-to-Prune Channels for ConvNets via Batch Whitening
Wenqi Shao, Hang Yu, Zhaoyang Zhang +3
This work presents a probabilistic channel pruning method to accelerate Convolutional Neural Networks (CNNs). Previous pruning methods often zero out unimportant channels in traini…
FAT: Learning Low-Bitwidth Parametric Representation via Frequency-Aware Transformation
Chaofan Tao, Rui Lin, Quan Chen +3
Learning convolutional neural networks (CNNs) with low bitwidth is challenging because performance may drop significantly after quantization. Prior arts often discretize the networ…
AdaX: Adaptive Gradient Descent with Exponential Long Term Memory
Wenjie Li, Zhaoyang Zhang, Xinjiang Wang +1
Although adaptive optimization algorithms such as Adam show fast convergence in many machine learning tasks, this paper identifies a problem of Adam by analyzing its performance in…
Differentiable Learning-to-Group Channels via Groupable Convolutional Neural Networks
Zhaoyang Zhang, Jingyu Li, Wenqi Shao +4
Group convolution, which divides the channels of ConvNets into groups, has achieved impressive improvement over the regular convolution operation. However, existing models, eg. Res…
Temporal Sequence Distillation: Towards Few-Frame Action Recognition in Videos
Zhaoyang Zhang, Zhanghui Kuang, Ping Luo +2
Video Analytics Software as a Service (VA SaaS) has been rapidly growing in recent years. VA SaaS is typically accessed by users using a lightweight client. Because the transmissio…
Boosting up Scene Text Detectors with Guided CNN
Xiaoyu Yue, Zhanghui Kuang, Zhaoyang Zhang +4
Deep CNNs have achieved great success in text detection. Most of existing methods attempt to improve accuracy with sophisticated network design, while paying less attention on spee…