In response to the problem that convolutional neural networks (CNNs) ignore the important information of the signals themselves, have weak decision-making mechanisms and single sensor information is not comprehensive, which limits the application of fault diagnosis technology based on CNN in high-performance scenarios, a rolling bearing fault diagnosis method using a multi-source wavelet convolution cross-fusion network (MSWCFN) was proposed. Firstly, vibration and acoustic signals of the rolling bearing operation were collected as the input of the multi-source signals, and wavelet knowledge was introduced to construct complex-valued wavelet convolution kernels to extract the time-frequency information carried by the signals. Secondly, the constructed cross-attention mechanism was used to perform feature fusion from multiple dimensions to obtain fault information of different sensitivities from multiple sources. Finally, the cosine embedding loss function was proposed to reduce the information differences between different features, and the classification results are output. The proposed model was experimentally verified. The results showed that without adding noise, the model could accurately identify various faults. When different levels of noise with different signal-to-noise ratios were added, the recognition rate of the model still exceeded 92%.