83 citations · 187 across the 14 of their papers we have counts for
14 papers
Accelerating Diffusion Models with One-to-Many Knowledge Distillation
Linfeng Zhang, Kaisheng Ma
Significant advancements in image generation have been made with diffusion models. Nevertheless, when contrasted with previous generative models, diffusion models face substantial…
Revisiting Data Augmentation in Model Compression: An Empirical and Comprehensive Study
Muzhou Yu, Linfeng Zhang, Kaisheng Ma
The excellent performance of deep neural networks is usually accompanied by a large number of parameters and computations, which have limited their usage on the resource-limited ed…
CORSD: Class-Oriented Relational Self Distillation
Muzhou Yu, Sia Huat Tan, Kailu Wu +3
Knowledge distillation conducts an effective model compression method while holding some limitations:(1) the feature based distillation methods only focus on distilling the feature…
Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?
Runpei Dong, Zekun Qi, Linfeng Zhang +5
The success of deep learning heavily relies on large-scale data with comprehensive labels, which is more expensive and time-consuming to fetch in 3D compared to 2D images or natura…
Structured Knowledge Distillation Towards Efficient and Compact Multi-View 3D Detection
Linfeng Zhang, Yukang Shi, Hung-Shuo Tai +4
Detecting 3D objects from multi-view images is a fundamental problem in 3D computer vision. Recently, significant breakthrough has been made in multi-view 3D detection tasks. Howev…
Contrastive Deep Supervision
Linfeng Zhang, Xin Chen, Junbo Zhang +2
The success of deep learning is usually accompanied by the growth in neural network depth. However, the traditional training method only supervises the neural network at its last l…
Region-aware Knowledge Distillation for Efficient Image-to-Image Translation
Linfeng Zhang, Xin Chen, Runpei Dong +1
Recent progress in image-to-image translation has witnessed the success of generative adversarial networks (GANs). However, GANs usually contain a huge number of parameters, which…
PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection
Linfeng Zhang, Runpei Dong, Hung-Shuo Tai +1
The remarkable breakthroughs in point cloud representation learning have boosted their usage in real-world applications such as self-driving cars and virtual reality. However, thes…
Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation
Linfeng Zhang, Xin Chen, Xiaobing Tu +3
Remarkable achievements have been attained with Generative Adversarial Networks (GANs) in image-to-image translation. However, due to a tremendous amount of parameters, state-of-th…
Fine-Grained Emotion Classification of Chinese Microblogs Based on Graph Convolution Networks
Yuni Lai, Linfeng Zhang, Donghong Han +2
Microblogs are widely used to express people's opinions and feelings in daily life. Sentiment analysis (SA) can timely detect personal sentiment polarities through analyzing text.…
Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
Xiaolong Ma, Sheng Lin, Shaokai Ye +10
Large deep neural network (DNN) models pose the key challenge to energy efficiency due to the significantly higher energy consumption of off-chip DRAM accesses than arithmetic or S…
SCAN: A Scalable Neural Networks Framework Towards Compact and Efficient Models
Linfeng Zhang, Zhanhong Tan, Jiebo Song +3
Remarkable achievements have been attained by deep neural networks in various applications. However, the increasing depth and width of such models also lead to explosive growth in…
Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation
Linfeng Zhang, Jiebo Song, Anni Gao +3
Convolutional neural networks have been widely deployed in various application scenarios. In order to extend the applications' boundaries to some accuracy-crucial domains, research…
StructADMM: A Systematic, High-Efficiency Framework of Structured Weight Pruning for DNNs
Tianyun Zhang, Shaokai Ye, Kaiqi Zhang +8
Weight pruning methods of DNNs have been demonstrated to achieve a good model pruning rate without loss of accuracy, thereby alleviating the significant computation/storage require…