结合多尺度融合与注意力机制的混合Transformer注视估计模型

彭黄果 ,  陈亮

科学技术与工程 ›› 2025, Vol. 25 ›› Issue (29) :12579-12585

PDF (4079KB)
科学技术与工程 ›› 2025, Vol. 25 ›› Issue (29) :12579-12585 DOI: 10.12404/j.issn.1671-1815.2408034
自动化技术、计算机技术

结合多尺度融合与注意力机制的混合Transformer注视估计模型

作者信息 +

Hybrid Transformer Gaze Estimation Model Combining Multi-scale Fusion and Attention Mechanism

Author information +
文章历史 +

摘要

基于外观的注视估计旨在从人脸图像中推测眼睛的注视方向,目前的模型在处理人脸图像时,难以提取局部和全局信息,容易受到背景、面部外貌等噪声干扰,导致预测精度下降。为此,提出了一种结合多尺度特征聚合和共享注意力机制的混合Transformer模型。首先通过CBAM(convolutional block attention module)通道空间注意力结合ResNet进行初步特征提取,以增强模型对双眼区域的敏感性;接着,通过多尺度特征融合模块整合不同尺度的信息,在捕捉头部姿势和面部表情等粗粒度特征的同时,关注眼部等细粒度细节,提高模型对多层次特征的表达能力。为提取全局信息,使用添加位置编码的Transformer模块,以捕获图像中的全局依赖关系并建模空间位置信息,使模型能够更全面理解图像的结构和布局。最后,引入改进的共享注意力,利用图像之间的相似性减少对无关特征的关注。实验结果表明,提出的模型在MPIIFaceGaze和RT-Gene数据集上表现最好,平均角度误差分别降低0.11°和0.12°,显示出在注视估计任务中的显著优势。

Abstract

Appearance-based gaze estimation aims to infer the gaze direction of the eyes from facial images. Current models face challenges in extracting both local and global information from facial images, which is often disrupted by background noise and facial appearance, leading to reduced prediction accuracy. To address this, a hybrid Transformer model was proposed, combining multi-scale feature aggregation with a shared attention mechanism. Preliminary feature extraction was performed using convolutional block attention module(CBAM) channel-space attention in combination with ResNet to enhance the model's sensitivity to the eye region. A multi-scale feature fusion module was introduced to integrate information at different scales, capturing coarse-grained features, such as head pose and facial expressions, while focusing on fine-grained details like the eyes. This improves the model's ability to express multi-level features. To extract global information, a Transformer module with positional encoding was used to capture global dependencies in the images and model spatial information, allowing for a more comprehensive understanding of the structure and layout of the images. An improved shared attention mechanism was then employed to reduce the focus on irrelevant features by leveraging the similarity between images. Experimental results show that the proposed model achieves outstanding performance on the Gaze360, MPIIFaceGaze, and RT-Gene datasets, with an average angular error reduction of 0.11° and 0.12°, respectively, demonstrating a significant advantage in gaze estimation tasks.

关键词

注视估计 / 多尺度特征 / 注意力 / Transformer

Key words

gaze estimation / multi-scale features / attention / Transformer

引用本文

引用格式 ▾
彭黄果, 陈亮. 结合多尺度融合与注意力机制的混合Transformer注视估计模型[J]. 科学技术与工程, 2025, 25(29): 12579-12585 DOI:10.12404/j.issn.1671-1815.2408034
PENG Huang-guo, CHEN Liang. Hybrid Transformer Gaze Estimation Model Combining Multi-scale Fusion and Attention Mechanism[J]. Science Technology and Engineering, 2025, 25(29): 12579-12585 DOI:10.12404/j.issn.1671-1815.2408034

登录浏览全文

4963

注册一个新账户 忘记密码

参考文献

基金资助

陕西省教育厅重点科学研究计划(22JS021)

PDF (4079KB)

39

访问

0

被引

详细

导航
相关文章

/

〈 〉