Most of the current electronic signature handwriting recognition models mainly focus on time series features and ignore the correlation between dimensions, and the feature modeling ability of time series features is insufficient. To solve this problem, an electronic signature handwriting recognition model based on double-branch Transformer was proposed to realize high-precision recognition of electronic signature handwriting. Firstly, a dual-branch feature extraction structure of Transformer is designed to fully capture temporal features and channel features; secondly, in order to realize multi-scale temporal feature learning, a multi-scale feature extractor is introduced in the temporal feature branch to enhance the encoder''s fine-grained learning of temporal features, and a feature recalibration module is introduced in the encoder to strengthen the learning of key points in the signature time sequence; in addition, a channel attention module encoder with a dual-view pooling mechanism is proposed in the channel feature branch to enhance the correlation between features of different dimensions; finally, a dual-branch feature fusion module is designed to effectively fuse complementary temporal features and channel features using a hierarchical design of local enhancement first and global fusion later. The performance evaluation is carried out on the public datasets MSDS and MCTY. The EER of the simulation scene 1vs1 achieves 2.54%and 2.61%respectively, and the EER of the simulation scene 4vs1 achieves 1.64%and 1.87%respectively. The experimental results show that the proposed method has higher recognition accuracy and has certain advantages in complex scenes.