In order to improve the poor accuracy of steel surface defects caused by poor detection of small targets, on the basis of the YOLOv5s, by adding the SE (attention mechanism) in the backbone network mechanism, C2f module instead of C3 module, the BIFPN (bidirectional feature pyramid network) instead of the PAN (path aggregation network) network in the neck, these three methods were used to investigate the improvement of the ability to the defect of small target detection. It aims to improve the detection accuracy and achieve real-time detection. The results show that the mAP (mean average precision) of the improved YOLOv5s-SCB algorithm on NEU-DET (northeastern university-detect) reaches 77.9%, which is 3.7% higher than that of the YOLOv5s network on the premise of real-time detection. Compared with other improved algorithms based on YOLOv5s and YOLOv8, YOLOv5S-SCB achieves better detection effect. It is concluded that the proposed steel surface defect detection algorithm YOLOv5S-SCB can better detect defects on steel surfaces.