Local and Global Semantic Fusion for Image-text Retrieval

Zhong-mei WANG , Shen-ao PENG , Jian-hua LIU , Wen-xiu AO , Quan-ming LI

Science Technology and Engineering ›› 2026, Vol. 26 ›› Issue (13) : 5574 -5582.

PDF
Science Technology and Engineering ›› 2026, Vol. 26 ›› Issue (13) : 5574-5582 DOI: 10.12404/j.issn.1671-1815.2503447
research-article
Local and Global Semantic Fusion for Image-text Retrieval
Author information +
History +
PDF

Abstract

Image-text retrieval serves as a pivotal bridge between vision and language, playing a critical role in achieving high-quality semantic alignment across modalities. To enhance the fine-grained discriminative capacity of semantic features in cross-modal retrieval, a multi-modal alignment framework that integrates intra-modal fine-grained local semantics with inter-modal global semantic correlations was proposed. Original features of images and texts were first extracted, followed by region-level relational reasoning through graph attention networks (GAT) to obtain relation-enhanced local features. An intra-modal attention mechanism was then employed to facilitate semantic interactions among samples within the same modality, thereby capturing comprehensive intra-modal associations and producing semantically enriched embeddings for both images and texts. Finally, a triangular constraint-enhanced triplet loss function was designed to optimize the alignment process. Experimental results on the MS-COCO dataset (1K test set) demonstrate improvements of 6.4% and 1.3% in image retrieval and text retrieval tasks, respectively, in terms of Recall@1, confirming the effectiveness of the proposed method. Experimental results on the MS-COCO dataset (1K test set) indicate that the proposed method improves Recall@1 (the percentage of queries for which at least one ground-truth match is retrieved in the top-1 results) by 6.4% and 1.3% on image-to-text and text-to-image retrieval, respectively, demonstrating its effectiveness.

Graphical abstract

Keywords

cross-modal image-text retrieval / relational reasoning / global matching / triplet loss

Cite this article

Download citation ▾
Zhong-mei WANG, Shen-ao PENG, Jian-hua LIU, Wen-xiu AO, Quan-ming LI. Local and Global Semantic Fusion for Image-text Retrieval. Science Technology and Engineering, 2026, 26(13): 5574-5582 DOI:10.12404/j.issn.1671-1815.2503447

登录浏览全文

4963

注册一个新账户 忘记密码

References

Funding

PDF (7722KB)

23

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉