The traditional detection of rust defects on segmental beams relies on manual identification, which poses high safety risks, low detection efficiency, and unstable accuracy. In order to solve these problems, a segment beam rust detection algorithm based on improved YOLOv5s was proposed, and the attention mechanism CBAM (convolutional block attention module) of channel attention and spatial attention fusion was added to the backbone network module to enhance the ability to extract features. The bi-directional weighted feature pyramid network structure BiFPN(bi-directional feature pyramid networ) was integrated into the neck network module to improve the network' s detection ability. The efficient intersection over union loss EIoU was combined with Alpha-IoU, which can perform power operation on the loss function, to generate an Alpha-EIoU Loss function to replace the original complete intersection over union loss CIoU, reduce the loss value and further improve the overall performance of the model. After experiments, it has been shown that the improved algorithm outperforms the original YOLOv5s algorithm in terms of accuracy, recall mAP@0. 5 and mAP@0. 5∶0.95 have increased by 2. 8% , 3. 0% , 2. 0% , and 5. 4% respectively, without adding too many parameters. After the validation of a virtual scene designed by the 3D modeling software 3dsMax, this algorithm can achieve high recognition accuracy in various backgrounds and has strong robustness. It has important theoretical significance and engineering value for the deployment of rust detection in prefabricated segmental beams.