Signal processing and deep learning are often combined to achieve better diagnostic results in the field of fault diagnosis. Based on this, the symplectic geometric mode decomposition was improved and the ResNeXt neural network was optimized, and then a gearbox fault diagnosis model was proposed based on the combination of optimized symplectic geometric mode decomposition and ResNeXt neural network was improved. Firstly, the collected vibration signals were filtered and reconstructed by optimized symplectic geometric mode decomposition to obtain the effective components. Then it was sent to the improved ResNeXt neural network for fault recognition and classification. The rolling bearing variable condition data from the University of Ottawa was used to verify the feasibility of the model. The gearbox data from drivetrain dynamics simula (DDS) was used for contrast experiment and anti-noise experiment, which verified the effectiveness of changes and the generalization of the model.