A semantic segmentation-based method for defect segmentation on thin strip cast and rolled steel plates was proposed to accurately and quickly identify surface defects. Firstly, defect images from the production line were annotated using Labeling software to create a defect segmentation dataset. Secondly, a TransUNet network model was established to recognize and segment surface defects, integrating an optimized DANet dual-attention fusion network to enhance model segmentation performance. Finally, comparative experiments between the improved model and other segmentation models were designed. The feasibility and effectiveness of the proposed method are verified through analysis of experimental results and evaluation metrics. The experiments demonstrate that the improved network achieves a segmentation accuracy of 96.85%, an average intersection over union of 96.99%, and a similarity coefficient of 92.98% for foreign object defects on thin strip cast and rolled steel plates, respectively increasing by 1.19%, 0.61%, and 0.63% compared to the TransUNet network. Additionally, the improved network achieves a segmentation accuracy of 92.86% on the publicly available hot-rolled strip steel defect dataset, indicating its versatility and providing technical guidance for intelligent detection of surface defects on steel plates.