基于局部特征重建的素描人脸识别方法

吕博千 ,  郭亚男 ,  杜康宁 ,  曹林 ,  孙文文

科学技术与工程 ›› 2025, Vol. 25 ›› Issue (24) :10313-10322

PDF (8507KB)
科学技术与工程 ›› 2025, Vol. 25 ›› Issue (24) :10313-10322 DOI: 10.12404/j.issn.1671-1815.2406741
自动化技术、计算机技术

基于局部特征重建的素描人脸识别方法

作者信息 +

A Sketch Face Recognition Method Based on Local Feature Reconstruction

Author information +
文章历史 +

摘要

针对素描人脸识别中光学与素描图像模态差异引起的局部特征不对齐问题,提出一种基于局部特征重建的素描人脸识别方法,该方法利用映射为可学习文本标记的人脸标签拉近匹配的图像对,提升模型识别效果。引入无参局部特征对齐模块,其借助于一种模态的局部特征无参重建另一种模态图像的人脸区域,实现更精准的跨模态局部特征对齐。为充分利用视觉语言预训练的跨模态描述能力,引入两阶段训练策略。在第一阶段,固定图像和文本编码器,仅优化文本标记;在第二阶段,固定特定于人脸标签的文本标记及其编码器,仅微调图像编码器。实验结果表明:在3个公共素描人脸数据集上,所提方法优于目前主流的方法。

Abstract

To address the issue of local feature misalignment caused by modality differences between photo and sketch images in sketch face recognition, a novel method based on local feature reconstruction was proposed. This approach leveraged face labels mapped to learnable text tokens to reduce the distance between matched image pairs, thereby enhancing the model recognition performance. A parameter-free local feature alignment module was introduced, which reconstructed the facial region of one modality using the local features of the other modality, without relying on external references, to achieve more precise cross-modal local feature alignment. In order to fully utilize the cross-modal description capability of visual language pre-training, a two-stage training strategy was proposed. In the first stage, the image and text encoders were frozen, and only the text tokens were optimized. In the second stage, the text tokens specific to face labels and their encoders were kept fixed, and only the image encoder was fine-tuned. Experimental results show that the proposed method outperforms current mainstream approaches on three public sketch face datasets.

关键词

多模态 / 人脸识别 / 局部特征对齐 / 文本提示

Key words

multi-modality / face recognition / local feature alignment / text prompt

引用本文

引用格式 ▾
吕博千, 郭亚男, 杜康宁, 曹林, 孙文文. 基于局部特征重建的素描人脸识别方法[J]. 科学技术与工程, 2025, 25(24): 10313-10322 DOI:10.12404/j.issn.1671-1815.2406741
LÜ Bo-qian, GUO Ya-nan, DU Kang-ning, CAO Lin, SUN Wen-wen. A Sketch Face Recognition Method Based on Local Feature Reconstruction[J]. Science Technology and Engineering, 2025, 25(24): 10313-10322 DOI:10.12404/j.issn.1671-1815.2406741

登录浏览全文

4963

注册一个新账户 忘记密码

参考文献

基金资助

国家自然科学基金(62201066)

北京市教委科研计划(KZ202111232049)

PDF (8507KB)

38

访问

0

被引

详细

导航
相关文章

/

〈 〉