
科学技术与工程

中国技术经济学会
基于局部特征重建的素描人脸识别方法
A Sketch Face Recognition Method Based on Local Feature Reconstruction
针对素描人脸识别中光学与素描图像模态差异引起的局部特征不对齐问题,提出一种基于局部特征重建的素描人脸识别方法,该方法利用映射为可学习文本标记的人脸标签拉近匹配的图像对,提升模型识别效果。引入无参局部特征对齐模块,其借助于一种模态的局部特征无参重建另一种模态图像的人脸区域,实现更精准的跨模态局部特征对齐。为充分利用视觉语言预训练的跨模态描述能力,引入两阶段训练策略。在第一阶段,固定图像和文本编码器,仅优化文本标记;在第二阶段,固定特定于人脸标签的文本标记及其编码器,仅微调图像编码器。实验结果表明:在3个公共素描人脸数据集上,所提方法优于目前主流的方法。
To address the issue of local feature misalignment caused by modality differences between photo and sketch images in sketch face recognition, a novel method based on local feature reconstruction was proposed. This approach leveraged face labels mapped to learnable text tokens to reduce the distance between matched image pairs, thereby enhancing the model recognition performance. A parameter-free local feature alignment module was introduced, which reconstructed the facial region of one modality using the local features of the other modality, without relying on external references, to achieve more precise cross-modal local feature alignment. In order to fully utilize the cross-modal description capability of visual language pre-training, a two-stage training strategy was proposed. In the first stage, the image and text encoders were frozen, and only the text tokens were optimized. In the second stage, the text tokens specific to face labels and their encoders were kept fixed, and only the image encoder was fine-tuned. Experimental results show that the proposed method outperforms current mainstream approaches on three public sketch face datasets.
multi-modality / face recognition / local feature alignment / text prompt
/
| 〈 |
|
〉 |