In order to solve the problem of poor robustness caused by the influence of motion speed, illumination, occlusion, complex background and so on, the adaptive recognition of decomposed action images of aerobics is studied by feature extraction method. Video sequence is divided into several segments by time energy pyramid, and the result shows that the action is not all aerobics action, which contains a lot of interference information. The human body target is extracted by background subtraction method, and then processed to get the binary image sequence of human body contour, and the outline of the outside world is obtained. The ratio of rectangular width to height is used to obtain the key frames according to the ratio of width to height. The optical flow between adjacent frames is solved by Laplace method to reduce the influence of background clutter. Feature vectors are extracted for key frames, and the decomposed action images and features are matched by similarity detection. The similarity threshold is set, and the images whose similarity is higher than the threshold are taken as recognition results. The results show that the proposed method has a high recognition accuracy of single aerobics video database, but there is only a certain degree of confusion. Compared with other methods, the proposed method has the highest recognition accuracy of complex database with different scenes. It can be seen that the proposed method is less disturbed by external environment and can guarantee high recognition accuracy.