Multi-modal Image Processing based on Coupled Dictionary Learning

Abstract

In real-world scenarios, many data processing problems often involve heterogeneous images associated with different imaging modalities. Since these multimodal images originate from the same phenomenon, it is realistic to assume that they share common attributes or characteristics. In this paper, we propose a multi-modal image processing framework based on coupled dictionary learning to capture similarities and disparities between different image modalities. In particular, our framework can capture favorable structure similarities across different image modalities such as edges, corners, and other elementary primitives in a learned sparse transform domain, instead of the original pixel domain, that can be used to improve a number of image processing tasks such as denoising, inpainting, or super-resolution. Practical experiments demonstrate that incorporating multimodal information using our framework brings notable benefits.

arXiv commentSPAWC 2018, 19th IEEE International Workshop On Signal Processing Advances In Wireless Communications

References in corpus (4)

• Multimodal Task-Driven Dictionary Learning for Image Classification
• Classification and Reconstruction of High-Dimensional Signals from Low-Dimensional Features in the Presence of Side Information
• Multi-modal dictionary learning for image separation with application in art investigation
• X-ray image separation via coupled dictionary learning

Cited by in corpus (1)

• Adaptive Image Sampling using Deep Learning and its Application on X-Ray Fluorescence Image Reconstruction