To address the challenges in existing multi-sensor data fusion methods for rail surface defect diagnosis, particularly their limitations in fully exploiting potential synergistic information among multimodal data and effectively bridging the semantic gap between multi-source heterogeneous data, a diagnostic method based on a progressive joint representation graph attention fusion network (PJR-GAFN) was proposed. Firstly, shared and specific autoencoders were employed to extract joint representations of multimodal features through shared and modality-specific representations. Secondly, a squeeze-and-excitation module was implemented to amplify defect-related features while suppressing non-essential characteristics. Thirdly, a progressive fusion module was introduced to facilitate comprehensive utilization of cross-modal synergistic information during feature extraction. Then, a source domain classifier and domain discriminator were incorporated to capture modality-invariant features across different modalities. Finally, the spatial attention aggregation properties of graph attention networks were leveraged to fuse multimodal features, thereby fully exploiting contextual semantic information. Experimental validation conducted on real-world rail surface defect datasets from domestic railway lines demonstrates that the proposed method achieves 95% diagnostic accuracy, confirming its effectiveness in rail surface defect detection.