4 papers
Group-Shared Low-Rank Approximation for Mobile-Efficient Pointwise Convolutions in Large-Kernel CNNs
Hao Luo, Yiting Yang, Wenyi Zhao +10
Large-kernel Convolutional Neural Networks (CNNs) deliver remarkable performance in vision tasks by significantly expanding receptive fields, yet their quadratic parameter growth c…
Efficient Adaptation of Pre-trained Vision Transformer via Householder Transformation
Wei Dong, Yuan Sun, Yiting Yang +7
A common strategy for Parameter-Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViTs) involves adapting the model to downstream tasks by learning a low-rank adapta…
Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approach
Wei Dong, Xing Zhang, Bihui Chen +5
Parameter-efficient fine-tuning for pre-trained Vision Transformers aims to adeptly tailor a model to downstream tasks by learning a minimal set of new adaptation parameters while…
Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing
Wei Dong, Dawei Yan, Zhijun Lin +1
The advent of high-capacity pre-trained models has revolutionized problem-solving in computer vision, shifting the focus from training task-specific models to adapting pre-trained…