To address the issues of distortion and poor segmentation results in weld defect image segmentation, the crack and porosity welding defect images in the rim production process were taken as the research object. An improved particle swarm optimization algorithm based on simulated annealing is proposed for the three-threshold image segmentation of welding defects. First, a three-dimensional Otsu model is constructed using the grayscale value, average grayscale value, and median grayscale value of the image. Next, an adaptive inertia weight and asymmetric learning factor were introduced and integrated into the SA strategy to enhance the algorithm's solving efficiency and ability to escape local optima. Finally, the SA-IPSO algorithm was used to optimize the three-dimensional Otsu model to obtain the optimal threshold and corresponding defect segmentation image. Various algorithms and models are employed to segment welding defect images. The results show that for crack and porosity defect images, the proposed improved algorithm outperforms the comparison algorithms in terms of peak signal-to-noise ratio and structural similarity evaluation metrics. The proposed method accelerates algorithm convergence while preventing distortion in segmentation results, thereby improving segmentation accuracy.