To address the challenges related to high costs associated with acquiring data of cracks on surface of underwater concrete structures, as well as the issues of significant noise interference and low identification accuracy, a novel crack identification model called moment feature transfer network (MTNet) based on image moment theory and transfer learning is proposed. To mitigate the limitations caused by insufficient data leading to overfitting and to enhance the identification accuracy of cracks of underwater concrete structures, several key contributions are made. Firstly, a multi-scale moment feature algorithm is introduced to extract both shape and texture information of cracks, thereby reducing the influence of background noise on crack identification. Secondly, a moment feature embedding module is developed to effectively integrate the multi-scale moment feature algorithm into the neural network model. The module facilitates the seamless incorporation of the algorithm into the model, ensuring enhanced identification performance. Furthermore, the MTNet model is presented for crack identification, combining the moment information of features and an attention module. This integration not only enables the extraction of semantic information from crack features but also suppresses background noise, resulting in improved crack segmentation quality on underwater concrete structures. Lastly, a dedicated underwater concrete crack dataset is established as the target domain dataset for transfer learning, significantly alleviating the impact of inadequate samples on identification performance. Experimental results validate the efficacy of the proposed MTNet model, demonstrating its remarkable identification accuracy and generalization performance, showing that the model is able to effectively identify cracks on surface of underwater concrete structures, even in complex backgrounds.