
科学技术与工程

中国技术经济学会
Local and Global Semantic Fusion for Image-text Retrieval
Zhong-mei WANG , Shen-ao PENG , Jian-hua LIU , Wen-xiu AO , Quan-ming LI
Science Technology and Engineering ›› 2026, Vol. 26 ›› Issue (13) : 5574 -5582.
Image-text retrieval serves as a pivotal bridge between vision and language, playing a critical role in achieving high-quality semantic alignment across modalities. To enhance the fine-grained discriminative capacity of semantic features in cross-modal retrieval, a multi-modal alignment framework that integrates intra-modal fine-grained local semantics with inter-modal global semantic correlations was proposed. Original features of images and texts were first extracted, followed by region-level relational reasoning through graph attention networks (GAT) to obtain relation-enhanced local features. An intra-modal attention mechanism was then employed to facilitate semantic interactions among samples within the same modality, thereby capturing comprehensive intra-modal associations and producing semantically enriched embeddings for both images and texts. Finally, a triangular constraint-enhanced triplet loss function was designed to optimize the alignment process. Experimental results on the MS-COCO dataset (1K test set) demonstrate improvements of 6.4% and 1.3% in image retrieval and text retrieval tasks, respectively, in terms of Recall@1, confirming the effectiveness of the proposed method. Experimental results on the MS-COCO dataset (1K test set) indicate that the proposed method improves Recall@1 (the percentage of queries for which at least one ground-truth match is retrieved in the top-1 results) by 6.4% and 1.3% on image-to-text and text-to-image retrieval, respectively, demonstrating its effectiveness.
cross-modal image-text retrieval / relational reasoning / global matching / triplet loss
/
| 〈 |
|
〉 |