Weak fault characteristics, strong non-stationarity, and limited representational capability of single-modality information remain significant challenges in high-speed train axle box bearing fault diagnosis under complex operating conditions. To address these issues, a cross-modal attention-driven multi-modal fault diagnosis method was constructed. A multi-scale convolutional structure was adopted to enhance local and global feature representation capabilities. A cross-modal attention mechanism was introduced to model correlations among different modalities and enable information interaction. Furthermore, a hierarchical fusion strategy was employed to progressively integrate multi-modal features and construct a more comprehensive feature representation. In addition, a gating mechanism was incorporated to emphasize feature components that are more relevant to fault identification. Experimental analyses were conducted using two sets of high-speed train axle box bearing fault vibration datasets. The results show that the proposed method achieves 100% diagnostic accuracy on both datasets and improves the average diagnostic accuracy by approximately 5% compared with advanced intelligent diagnosis methods. These results demonstrate the effectiveness, robustness, and superior diagnostic performance of the proposed method for high-speed train axle box bearing fault identification and classification.