Large Kernel Distillation Network for Efficient Single Image Super-Resolution
arXiv:2407.14340 · doi:10.1109/CVPRW59228.2023.00135
Abstract
Efficient and lightweight single-image super-resolution (SISR) has achieved remarkable performance in recent years. One effective approach is the use of large kernel designs, which have been shown to improve the performance of SISR models while reducing their computational requirements. However, current state-of-the-art (SOTA) models still face problems such as high computational costs. To address these issues, we propose the Large Kernel Distillation Network (LKDN) in this paper. Our approach simplifies the model structure and introduces more efficient attention modules to reduce computational costs while also improving performance. Specifically, we employ the reparameterization technique to enhance model performance without adding extra cost. We also introduce a new optimizer from other tasks to SISR, which improves training speed and performance. Our experimental results demonstrate that LKDN outperforms existing lightweight SR methods and achieves SOTA performance.
Accepted to CVPR workshop 2023
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Gaussian Error Linear Units (GELUs)
- On the Convergence of Adam and Beyond
- Sketch-based Manga Retrieval using Manga109 Dataset
- Lightweight Image Super-Resolution with Information Multi-distillation Network
- Rethinking the Value of Network Pruning
- The Marginal Value of Adaptive Gradient Methods in Machine Learning
- Improving Generalization Performance by Switching from Adam to SGD
- Rigid-Motion Scattering for Texture Classification
- More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity
- Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models