Due to the influence of the skylight time and the natural environment, the traditional crack detection method is easy to confuse the cracks of the track plate with complex background noises. The number of crack image samples of the track plate is often unable to meet the requirements of the training model. It eventually leads to detection errors and omissions. In order to improve the efficiency of crack detection for track plate in complex environment and small samples, a crack detection model of track plate based on convolutional neural network and generated adversarial network was proposed. Firstly, the data set of the track plate crack was propuced. The image data set of the track plate crack disease was enhanced by cycle generative adversarial network. Then, the expansion convolution and coordinate attention mechanism modules were added to the ResNet50 network to deepen the network layer. It can obtain more abundant image features and enhance network's ability to learn key features. Finally, the performance of the improved ResNet50 network model was compared with three existing depth convolutional neural network models. Its accuracy, precision, recall, and F₁ score are 93.51%, 93.44%, 93.46%, and 93.45%, respectively, which are all higher than other models. The model effectively solves the problems such as lack of track plate crack image and complex image background. The detection results meet the needs of actual track plate crack repair operations, which can provide reference for track structure maintenance personnel.