66 citations · 102 across the 9 of their papers we have counts for
9 papers
TwinQuant: Learnable Subspace Decomposition for 4-Bit LLM Quantization
Haodong Wang, Junjie Liu, Zicong Hong +4
4-bit quantization reduces the memory footprint and latency of large language model inference, but its aggressive precision reduction can severely degrade accuracy. Prior methods a…
Towards Accurate Post-Training Quantization for Vision Transformer
Yifu Ding, Haotong Qin, Qinghua Yan +4
Vision transformer emerges as a potential architecture for vision tasks. However, the intense computation and non-negligible delay hinder its application in the real world. As a wi…
Compressing Models with Few Samples: Mimicking then Replacing
Huanyu Wang, Junjie Liu, Xin Ma +3
Few-sample compression aims to compress a big redundant model into a small compact one with only few samples. If we fine-tune models with these limited few samples directly, models…
Condensation-Net: Memory-Efficient Network Architecture with Cross-Channel Pooling Layers and Virtual Feature Maps
Tse-Wei Chen, Motoki Yoshinaga, Hongxing Gao +5
"Lightweight convolutional neural networks" is an important research topic in the field of embedded vision. To implement image recognition tasks on a resource-limited hardware plat…
BAMSProd: A Step towards Generalizing the Adaptive Optimization Methods to Deep Binary Model
Junjie Liu, Dongchao Wen, Deyu Wang +4
Recent methods have significantly reduced the performance degradation of Binary Neural Networks (BNNs), but guaranteeing the effective and efficient training of BNNs is an unsolved…
QuantNet: Learning to Quantize by Learning within Fully Differentiable Framework
Junjie Liu, Dongchao Wen, Deyu Wang +4
Despite the achievements of recent binarization methods on reducing the performance degradation of Binary Neural Networks (BNNs), gradient mismatching caused by the Straight-Throug…
Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layers
Junjie Liu, Zhe Xu, Runbin Shi +2
We present a novel network pruning algorithm called Dynamic Sparse Training that can jointly find the optimal network parameters and sparse network structure in a unified optimizat…
DupNet: Towards Very Tiny Quantized CNN with Improved Accuracy for Face Detection
Hongxing Gao, Wei Tao, Dongchao Wen +4
Deploying deep learning based face detectors on edge devices is a challenging task due to the limited computation resources. Even though binarizing the weights of a very tiny netwo…
Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation
Junjie Liu, Dongchao Wen, Hongxing Gao +4
Despite the recent works on knowledge distillation (KD) have achieved a further improvement through elaborately modeling the decision boundary as the posterior knowledge, their per…