Improper sitting posture often leads to myopia, scoliosis and degeneration in adolescents. Research on intelligent monitoring technology that can quickly and accurately identify irregular sitting postures can help maintain correct posture and prevent health problems. In order to solve the problem that the RGB image is easily disturbed by the hint strength and occlusion factors and the recognition rate is not high, by using the dual-stream RGB-D image as the double input, the residual structure in the ResNet network was used to improve the EfficientNet network structure, and a dual-stream RGB-D multi-modal information fusion test based on the improved R-EfficientNet was proposed. The mean average precision (mAP) reaches 98.5%. Compared with different input methods such as CNN, Vgg16, ResNet18, EfficientNet, and RGB-D, the method proposed has achieved a higher recognition rate. The method not only can be used for monitoring the sitting posture, but also has a medical and sociological sitting posture recognition method. In addition, it also provides better automated work solutions for ergonomic people.