6 papers · 1 filter
S2HPruner: Soft-to-Hard Distillation Bridges the Discretization Gap in Pruning
Weihao Lin, Shengji Tang, Chong Yu +2
Recently, differentiable mask pruning methods optimize the continuous relaxation architecture (soft network) as the proxy of the pruned discrete network (hard network) for superior…
Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression
Hancheng Ye, Chong Yu, Peng Ye +5
Recent Vision Transformer Compression (VTC) works mainly follow a two-stage scheme, where the importance score of each model unit is first evaluated or preset in each submodule, fo…
Enhanced Sparsification via Stimulative Training
Shengji Tang, Weihao Lin, Hancheng Ye +4
Sparsification-based pruning has been an important category in model compression. Existing methods commonly set sparsity-inducing penalty terms to suppress the importance of droppe…
MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer
Jianjian Cao, Peng Ye, Shengze Li +4
Vision-Language Transformers (VLTs) have shown great success recently, but are meanwhile accompanied by heavy computation costs, where a major reason can be attributed to the large…
Efficient Architecture Search via Bi-level Data Pruning
Chongjun Tu, Peng Ye, Weihao Lin +5
Improving the efficiency of Neural Architecture Search (NAS) is a challenging but significant task that has received much attention. Previous works mainly adopted the Differentiabl…
SpVOS: Efficient Video Object Segmentation with Triple Sparse Convolution
Weihao Lin, Tao Chen, Chong Yu
Semi-supervised video object segmentation (Semi-VOS), which requires only annotating the first frame of a video to segment future frames, has received increased attention recently.…