Infrared and Visible Image Fusion using a Deep Learning Framework

Abstract

In recent years, deep learning has become a very active research tool which is used in many image processing fields. In this paper, we propose an effective image fusion method using a deep learning framework to generate a single image which contains all the features from infrared and visible images. First, the source images are decomposed into base parts and detail content. Then the base parts are fused by weighted-averaging. For the detail content, we use a deep learning network to extract multi-layer features. Using these features, we use l_1-norm and weighted-average strategy to generate several candidates of the fused detail content. Once we get these candidates, the max selection strategy is used to get final fused detail content. Finally, the fused image will be reconstructed by combining the fused base part and detail content. The experimental results demonstrate that our proposed method achieves state-of-the-art performance in both objective assessment and visual quality. The Code of our fusion method is available at https://github.com/hli1221/imagefusion_deeplearning

arXiv comment6 pages, 6 figures, 2 tables, ICPR 2018(accepted)

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Multi-focus Image Fusion using dictionary learning and Low-Rank Representation
• Image Fusion With Cosparse Analysis Operator

Cited by in corpus (28)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• RFN-Nest: An end-to-end residual fusion network for infrared and visible images
• NestFuse: An Infrared and Visible Image Fusion Architecture based on Nest Connection and Spatial/Channel Attention Models
• MDLatLRR: A novel decomposition method for infrared and visible image fusion
• Infrared and Visible Image Fusion with ResNet and zero-phase component analysis
• CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion
• Bayesian Fusion for Infrared and Visible Images
• Infrared and visible image fusion using Latent Low-Rank Representation
• Multi-focus Noisy Image Fusion using Low-Rank Representation
• Fast and Efficient Zero-Learning Image Fusion
• FIRe-GAN: A novel Deep Learning-based infrared-visible fusion method for wildfire imagery
• VIFB: A Visible and Infrared Image Fusion Benchmark
• Deep Convolutional Sparse Coding Networks for Image Fusion
• Efficient and Model-Based Infrared and Visible Image Fusion Via Algorithm Unrolling
• Visible and Infrared Image Fusion Using Encoder-Decoder Network
• AE-Net: Autonomous Evolution Image Fusion Method Inspired by Human Cognitive Mechanism
• Effective Data Fusion with Generalized Vegetation Index: Evidence from Land Cover Segmentation in Agriculture
• Deep Learning for HDR Imaging: State-of-the-Art and Future Trends
• FDCT: Frequency-Aware Decomposition and Cross-Modal Token-Alignment for Multi-Sensor Target Classification
• When Image Decomposition Meets Deep Learning: A Novel Infrared and Visible Image Fusion Method
• Non-linear and Selective Fusion of Cross-Modal Images
• Image fusion using symmetric skip autoencodervia an Adversarial Regulariser
• HyperFaceNet: A Hyperspectral Face Recognition Method Based on Deep Fusion
• Maritime Small Object Detection from UAVs using Deep Learning with Altitude-Aware Dynamic Tiling
• Semantic Image Fusion
• Cross-Modal Image Fusion Theory Guided by Subjective Visual Attention
• AE-Netv2: Optimization of Image Fusion Efficiency and Network Architecture
• Unsupervised Image Fusion Using Deep Image Priors