activity
20182023
most citedViTKD: Practical Guidelines for ViT feature knowledge distillation

23 citations · 146 across the 33 of their papers we have counts for

collaborators
Showing 2023Show all

12 papers · 1 filter

cs.CV2023★ 3 cited

Effective Whole-body Pose Estimation with Two-stages Distillation

Zhendong Yang, Ailing Zeng, Chun Yuan +1

Whole-body pose estimation localizes the human body, hand, face, and foot keypoints in an image. This task is challenging due to multi-scale body parts, fine-grained localization f…

cs.CV2023★ 17 cited

DreamDiffusion: Generating High-Quality Images from Brain EEG Signals

Yunpeng Bai, Xintao Wang, Yan-pei Cao +3

This paper introduces DreamDiffusion, a novel method for generating high-quality images directly from brain electroencephalogram (EEG) signals, without the need to translate though…

cs.CV2023

Towards Effective Collaborative Learning in Long-Tailed Recognition

Zhengzhuo Xu, Zenghao Chai, Chengyin Xu +2

Real-world data usually suffers from severe class imbalance and long-tailed distributions, where minority classes are significantly underrepresented compared to the majority ones.…

cs.LG2023

Learning to Learn from APIs: Black-Box Data-Free Meta-Learning

Zixuan Hu, Li Shen, Zhenyi Wang +3

Data-free meta-learning (DFML) aims to enable efficient learning of new tasks by meta-learning from a collection of pre-trained models without access to the training data. Existing…

cs.CV2023★ 5 cited

CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers

Dachuan Shi, Chaofan Tao, Anyi Rao +3

Recent vision-language models have achieved tremendous advances. However, their computational costs are also escalating dramatically, making model acceleration exceedingly critical…

cs.CL2023

Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation

Yuxin Ren, Zihan Zhong, Xingjian Shi +3

It has been commonly observed that a teacher model with superior performance does not necessarily result in a stronger student, highlighting a discrepancy between current teacher t…

cs.CV2023★ 1 cited

HMC: Hierarchical Mesh Coarsening for Skeleton-free Motion Retargeting

Haoyu Wang, Shaoli Huang, Fang Zhao +2

We present a simple yet effective method for skeleton-free motion retargeting. Previous methods transfer motion between high-resolution meshes, failing to preserve the inherent loc…

cs.CV2023★ 5 cited

From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels

Zhendong Yang, Ailing Zeng, Zhe Li +3

Knowledge Distillation (KD) uses the teacher's prediction logits as soft labels to guide the student, while self-KD does not need a real teacher to require the soft labels. This wo…

cs.CV2023★ 3 cited

HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details

Zenghao Chai, Tianke Zhang, Tianyu He +7

3D Morphable Models (3DMMs) demonstrate great potential for reconstructing faithful and animatable 3D facial surfaces from a single image. The facial surface is influenced by the c…

cs.LG2023

Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning

Zixuan Hu, Li Shen, Zhenyi Wang +3

The goal of data-free meta-learning is to learn useful prior knowledge from a collection of pre-trained models without accessing their training data. However, existing works only s…

cs.CV2023

Rethink Long-tailed Recognition with Vision Transformers

Zhengzhuo Xu, Shuo Yang, Xingjun Wang +1

In the real world, data tends to follow long-tailed distributions w.r.t. class or attribution, motivating the challenging Long-Tailed Recognition (LTR) problem. In this paper, we r…

cs.CV2023★ 6 cited

UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers

Dachuan Shi, Chaofan Tao, Ying Jin +3

Real-world data contains a vast amount of multimodal information, among which vision and language are the two most representative modalities. Moreover, increasingly heavier models,…