Towards Image Understanding from Deep Compression without Decoding

Abstract

Motivated by recent work on deep neural network (DNN)-based image compression methods showing potential improvements in image quality, savings in storage, and bandwidth reduction, we propose to perform image understanding tasks such as classification and segmentation directly on the compressed representations produced by these compression methods. Since the encoders and decoders in DNN-based compression methods are neural networks with feature-maps as internal representations of the images, we directly integrate these with architectures for image understanding. This bypasses decoding of the compressed representation into RGB space and reduces computational cost. Our study shows that accuracies comparable to networks that operate on compressed RGB images can be achieved while reducing the computational complexity up to $2\times$. Furthermore, we show that synergies are obtained by jointly training compression networks with classification networks on the compressed representations, improving image quality, classification accuracy, and segmentation performance. We find that inference from compressed representations is particularly advantageous compared to inference from compressed RGB images for aggressive compression rates.

arXiv commentICLR 2018 conference paper

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Lossy Image Compression with Compressive Autoencoders
• Variable Rate Image Compression with Recurrent Neural Networks
• Real-Time Adaptive Image Compression
• Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks
• Learning Convolutional Networks for Content-weighted Image Compression
• End-to-end optimization of nonlinear transform codes for perceptual quality
• On Hyperspectral Classification in the Compressed Domain

Cited by in corpus (25)

• Scalable Image Coding for Humans and Machines
• Deep Learning-Based Video Coding: A Review and A Case Study
• Deep Joint Source-Channel Coding for Wireless Image Retrieval
• Fast object detection in compressed JPEG Images
• DuetFace: Collaborative Privacy-Preserving Face Recognition via Channel Splitting in the Frequency Domain
• Improved Hybrid Layered Image Compression using Deep Learning and Traditional Codecs
• Compressing the Input for CNNs with the First-Order Scattering Transform
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Deep Generative Models for Distribution-Preserving Lossy Compression
• Image compression optimized for 3D reconstruction by utilizing deep neural networks
• Distributed Learning and Inference with Compressed Images
• AdaCompress: Adaptive Compression for Online Computer Vision Services
• DSSLIC: Deep Semantic Segmentation-based Layered Image Compression
• Faster and Accurate Classification for JPEG2000 Compressed Images in Networked Applications
• Harmonic Networks: Integrating Spectral Information into CNNs
• Observer Dependent Lossy Image Compression
• Machine Perception-Driven Image Compression: A Layered Generative Approach
• Towards Coding for Human and Machine Vision: A Scalable Image Coding Approach
• FreqNet: A Frequency-domain Image Super-Resolution Network with Dicrete Cosine Transform
• End-to-end Compression Towards Machine Vision: Network Architecture Design and Optimization
• Neural Communication Systems with Bandwidth-limited Channel
• Task-driven Semantic Coding via Reinforcement Learning
• Multimedia Edge Computing
• Progressive Compressed Records: Taking a Byte out of Deep Learning Data
• Place-specific Background Modeling Using Recursive Autoencoders