Dual-semantic-guided Fusion Framework for Infrared and Visible Images toward Advanced Vision Tasks

Shi-wei LI , Lin-bo TIAN , Kun LIU , Liu-zhou XIE , Jian-qiang WANG

Science Technology and Engineering ›› 2026, Vol. 26 ›› Issue (17) : 7365 -7388.

PDF
Science Technology and Engineering ›› 2026, Vol. 26 ›› Issue (17) : 7365-7388 DOI: 10.12404/j.issn.1671-1815.2507158
research-article
Dual-semantic-guided Fusion Framework for Infrared and Visible Images toward Advanced Vision Tasks
Author information +
History +
PDF

Abstract

In order to fully integrate the complementary information of infrared and visible images and improve the feature preservation and task adaptability of fusion results, an enhancement-feature and Transformer-integrated dual-semantic image fusion method (enhancement-feature transformer-integrated dual semantic image fusion, ETDFusion) was proposed to investigate multimodal image fusion. Feature enhancement modules were designed to strengthen the edge details of infrared images and the brightness information of visible images. A hybrid architecture combining convolutional neural network(CNN) and Vision Transformer was constructed to achieve coordinated optimization of local feature extraction and global dependency modeling. In addition, a dual-semantic loss function was introduced by incorporating semantic segmentation and object detection constraints, and a three-stage training strategy was employed to guide the network in learning fusion features with strong semantic structures. The results show that the proposed method produces fused images with higher visual quality in terms of detail preservation and target saliency, and achieves superior performance to existing algorithms on multiple subjective and objective metrics. It is concluded that the ETDFusion network effectively balances multimodal information complementation and high-level visual task adaptation, providing an engineering-feasible technical pathway for infrared and visible image fusion. It can be further applied to perception tasks in complex environments such as security monitoring, intelligent transportation, and autonomous driving.

Graphical abstract

Keywords

infrared and visible image fusion / feature enhancement / vision Transformer / dual-semantic guidance / task-driven evaluation

Cite this article

Download citation ▾
Shi-wei LI, Lin-bo TIAN, Kun LIU, Liu-zhou XIE, Jian-qiang WANG. Dual-semantic-guided Fusion Framework for Infrared and Visible Images toward Advanced Vision Tasks. Science Technology and Engineering, 2026, 26(17): 7365-7388 DOI:10.12404/j.issn.1671-1815.2507158

登录浏览全文

4963

注册一个新账户 忘记密码

References

Funding

PDF (48076KB)

32

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉