To address the challenges in existing multi-sensor data fusion methods for rail surface defect diagnosis, particularly their limitations in fully exploiting potential synergistic information among multimodal data and effectively bridging the semantic gap between multi-source heterogeneous data, a diagnostic method based on a progressive joint representation graph attention fusion network (PJR-GAFN) was proposed. Firstly, shared and specific autoencoders were employed to extract joint representations of multimodal features through shared and modality-specific representations. Secondly, a squeeze-and-excitation module was implemented to amplify defect-related features while suppressing non-essential characteristics. Thirdly, a progressive fusion module was introduced to facilitate comprehensive utilization of cross-modal synergistic information during feature extraction. Then, a source domain classifier and domain discriminator were incorporated to capture modality-invariant features across different modalities. Finally, the spatial attention aggregation properties of graph attention networks were leveraged to fuse multimodal features, thereby fully exploiting contextual semantic information. Experimental validation conducted on real-world rail surface defect datasets from domestic railway lines demonstrates that the proposed method achieves 95% diagnostic accuracy, confirming its effectiveness in rail surface defect detection.
WANG Zhong-mei, NIE Peng-xuan, LIU Jian-hua, WU Hai-bo, LI Quan-ming.
Rail Surface Defect Diagnosis Based on Image-vibration Multimodal Data Fusion[J].
Science Technology and Engineering, 2026, 26(9): 3955-3966 DOI:10.12404/j.issn.1671-1815.2503239
FengFenling, LiangAilan, HuangQiying, et al. Strategy analysis on railway insured transportation and freight insurance cooperative development based on Hotelling model[J]. Journal of Railway Science and Engineering, 2023, 20(1): 356-363.
YuHuijun, ZhangJinsheng, LiuJianhua, et al. Rail surface state recognition based on attention network and cost-sensitive learning under imbalanced data[J]. Science Technology and Engineering, 2024, 24(5): 1972-1979.
WangSiwei, YaoDechen, YangJianwei, et al. Detection method of rail corrugation in metro based on wavelet synchrosqueezing transform under variable speed[J]. Science Technology and Engineering, 2023, 23(16): 7080-7086.
[7]
ShenY, ZhongQ, ZhengS, et al. A multi-modal approach to rail surface condition analysis: the MFDF-Net[J]. IEEE Access, 2024, 12: 132480-132494.
[8]
ChenZ, WangQ, HeQ, et al. CUFuse: camera and ultrasound data fusion for rail defect detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 21971-21983.
[9]
LecunY, BengioY, HintonG. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[10]
SchmidhuberJ. Deep learning in neural networks: an overview[J]. Neural Networks, 2015, 61: 85-117.
[11]
GeX, CaoZ, QinY, et al. An anomaly detection method for railway track using semi-supervised learning and vision-LiDAR decision fusion[J]. IEEE Transactions on Instrumentation and Measure- ment, 2024, 73: 1-15.
[12]
YangJ, ZhouW, WuR, et al. CSANet: contour and semantic feature alignment fusion network for rail surface defect detection[J]. IEEE Signal Processing Letters, 2023, 30: 972-976.
[13]
WangQ, WangX, HeQ, et al. 3D tensor-based point cloud and image fusion for robust detection and measurement of rail surface defects[J]. Automation in Construction, 2024, 161: 105342.
[14]
PiaoY, JiW, LiJ, et al. Depth-induced multi-scale re-current attention network for saliency detection[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 7254-7263.
ZhaoHongwei, ZhengJiajun, ZhaoXinxin, et al. Rail surface defect method based on bimodal-modal deep learning[J]. Computer Engineering and Applications, 2023, 59(7): 285-293.
[17]
LiuN, ZhangN, HanJ. Learning selective self-mutual attention for RGB-D saliency detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 13756-13765.
[18]
YangT, XuT, ChengY, et al. A fusion method based on 1D vibration signals and 2D images for detection of railway surface defects[C]// 2023 3rd International Conference on Neural Networks, Information and Communication Engineering (NNICE). Guangzhou: IEEE, 2023: 282-286.
[19]
HuJ, ShenL, SunG. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
ChoiJ Y, HanJ M. Deep learning (Fast R-CNN)-based evaluation of rail surface defects[J]. Applied Sciences, 2024, 14(5): 1874.
[22]
WangM, ZhouY. Autonomous rail surface defect identification based on an improved one-stage object detection algorithm[J]. Journal of Performance of Constructed Facilities, 2024, 38(5): 04024041.
[23]
RahmanM A, JamalS, TaheriH. Remote condition monitoring of rail tracks using distributed acoustic sensing (DAS): a deep CNN-LSTM-SW based model[J]. Green Energy and Intelligent Transportation, 2024, 3(5): 100178.
[24]
EunusS I, HossainS, RidwanA E M, et al. ECARRNet: an efficient LSTM-based ensembled deep neural network architecture for railway fault detection[J]. AI, 2024, 5(2): 482-503.
[25]
LessmeierC, KimothoJ K, ZimmerD, et al. Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: a benchmark data set for data-driven classification[C]// PHM Society European Conference. Dresden: PHM Society, 2016, 3(1): 1577.
PDF
(17054KB)
