5 papers
Decoder-Free Distillation for Quantized Image Restoration
S. M. A. Sharif, Abdur Rehman, Seongwan Kim +1
Quantization-Aware Training (QAT), combined with Knowledge Distillation (KD), holds immense promise for compressing models for edge deployment. However, joint optimization for prec…
LCSB: Layer-Cyclic Selective Backpropagation for Memory-Efficient On-Device LLM Fine-Tuning
Juneyoung Park, Eunbeen Yoon, Seongwan Kim. Jaeho Lee
Memory-efficient backpropagation (MeBP) has enabled first-order fine-tuning of large language models (LLMs) on mobile devices with less than 1GB memory. However, MeBP requires back…
Punching Above Precision: Small Quantized Model Distillation with Learnable Regularizer
Abdur Rehman, S M A Sharif, Md Abdur Rahaman +3
Quantization-aware training (QAT) combined with knowledge distillation (KD) is a promising strategy for compressing Artificial Intelligence (AI) models for deployment on resource-c…
Riemannian Optimization for LoRA on the Stiefel Manifold
Juneyoung Park, Minjae Kang, Seongbae Lee +3
While powerful, large language models (LLMs) present significant fine-tuning challenges due to their size. Parameter-efficient fine-tuning (PEFT) methods like LoRA provide solution…
The Tenth NTIRE 2025 Image Denoising Challenge Report
Lei Sun, Hang Guo, Bin Ren +91
This paper presents an overview of the NTIRE 2025 Image Denoising Challenge (σ = 50), highlighting the proposed methodologies and corresponding results. The primary objective is to…