
科学技术与工程

中国技术经济学会
Dual-semantic-guided Fusion Framework for Infrared and Visible Images toward Advanced Vision Tasks
Shi-wei LI , Lin-bo TIAN , Kun LIU , Liu-zhou XIE , Jian-qiang WANG
Science Technology and Engineering ›› 2026, Vol. 26 ›› Issue (17) : 7365 -7388.
In order to fully integrate the complementary information of infrared and visible images and improve the feature preservation and task adaptability of fusion results, an enhancement-feature and Transformer-integrated dual-semantic image fusion method (enhancement-feature transformer-integrated dual semantic image fusion, ETDFusion) was proposed to investigate multimodal image fusion. Feature enhancement modules were designed to strengthen the edge details of infrared images and the brightness information of visible images. A hybrid architecture combining convolutional neural network(CNN) and Vision Transformer was constructed to achieve coordinated optimization of local feature extraction and global dependency modeling. In addition, a dual-semantic loss function was introduced by incorporating semantic segmentation and object detection constraints, and a three-stage training strategy was employed to guide the network in learning fusion features with strong semantic structures. The results show that the proposed method produces fused images with higher visual quality in terms of detail preservation and target saliency, and achieves superior performance to existing algorithms on multiple subjective and objective metrics. It is concluded that the ETDFusion network effectively balances multimodal information complementation and high-level visual task adaptation, providing an engineering-feasible technical pathway for infrared and visible image fusion. It can be further applied to perception tasks in complex environments such as security monitoring, intelligent transportation, and autonomous driving.
infrared and visible image fusion / feature enhancement / vision Transformer / dual-semantic guidance / task-driven evaluation
/
| 〈 |
|
〉 |