Differentiable JPEG: The Devil is in the Details
arXiv:2309.06978 · doi:10.1109/WACV57701.2024.00408
Abstract
JPEG remains one of the most widespread lossy image coding methods. However, the non-differentiable nature of JPEG restricts the application in deep learning pipelines. Several differentiable approximations of JPEG have recently been proposed to address this issue. This paper conducts a comprehensive review of existing diff. JPEG approaches and identifies critical details that have been missed by previous methods. To this end, we propose a novel diff. JPEG approach, overcoming previous limitations. Our approach is differentiable w.r.t. the input image, the JPEG quality, the quantization tables, and the color conversion parameters. We evaluate the forward and backward performance of our diff. JPEG approach against existing methods. Additionally, extensive ablations are performed to evaluate crucial design choices. Our proposed diff. JPEG resembles the (non-diff.) reference implementation best, significantly surpassing the recent-best diff. approach by dB (PSNR) on average. For strong compression rates, we can even improve PSNR by dB. Strong adversarial attack results are yielded by our diff. JPEG, demonstrating the effective gradient approximation. Our code is available at https://github.com/necla-ml/Diff-JPEG.
arXiv commentAccepted at WACV 2024. Project page: https://christophreich1996.github.io/differentiable_jpeg/ WACV paper: https://openaccess.thecvf.com/content/WACV2024/html/Reich_Differentiable_JPEG_The_Devil_Is_in_the_Details_WACV_2024_paper.html
References in corpus (6)
- Neural Discrete Representation Learning
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Training Generative Adversarial Networks with Limited Data
- FaceGuard: Proactive Deepfake Detection
- Towards Robust Data Hiding Against (JPEG) Compression: A Pseudo-Differentiable Deep Learning Approach
- Differentiable Data Augmentation with Kornia