To address the challenges related to high costs associated with acquiring data of cracks on surface of underwater concrete structures, as well as the issues of significant noise interference and low identification accuracy, a novel crack identification model called moment feature transfer network (MTNet) based on image moment theory and transfer learning was proposed. To mitigate the limitations caused by insufficient data leading to overfitting and to enhance the identification accuracy of cracks of underwater concrete structures, several key contributions were made. Firstly, a multi-scale moment feature algorithm was introduced to extract both shape and texture information of cracks, thereby reducing the influence of background noise on crack identification. Secondly, a moment feature embedding module was developed to effectively integrate the multi-scale moment feature algorithm into the neural network model. The module facilitates the seamless incorporation of the algorithm into the model, ensuring enhanced identification performance. Furthermore, the MTNet model was presented for crack identification, combining the moment information of features and an attention module. This integration not only enabled the extraction of semantic information from crack features but also suppressed background noise, resulting in improved crack segmentation quality on underwater concrete structures. Lastly, a dedicated underwater concrete crack dataset was established as the target domain dataset for transfer learning, significantly alleviating the impact of inadequate samples on identification performance. Experimental results validate the efficacy of the proposed MTNet model, demonstrating its remarkable identification accuracy and generalization performance, showing that the model is able to effectively identify cracks on surface of underwater concrete structures, even in complex backgrounds.